Reporting AIC weights and model likelihoods

questions concerning analysis/theory using program MARK

Reporting AIC weights and model likelihoods

Postby brp » Mon Oct 11, 2010 1:10 pm

Page 4-40 of the MARK book states:
"From a practical standpoint, when reporting model selection results (in a paper, or report), it is
useful to report both AIC weights and either model likelihoods or evidence ratios (reporting both
would be redundant)."


The top model (when sorted by Delta AIC) always has a model likelihood of 1. Statistically speaking, what does a model-likelihood of 1 mean? Someone may think it means "We're 100% certain this is the correct model." but I know that's not right.

Is it correct to write in a report something like the following?

Using var1 as a covariate for the capture and/or recapture rates resulted in 3 models with model likelihoods of 1, 0.84, and 0.53 with a AICc weights of 0.32, 0.27, and 0.17. The cumulative AICc weight is 0.76, meaning 76% of the support in the data are for these three models.


Thank you,
Becky
brp
 
Posts: 26
Joined: Tue Sep 29, 2009 12:54 pm
Location: Bozeman, MT

Re: Reporting AIC weights and model likelihoods

Postby cooch » Mon Oct 11, 2010 1:17 pm

brp wrote:Page 4-40 of the MARK book states:
"From a practical standpoint, when reporting model selection results (in a paper, or report), it is
useful to report both AIC weights and either model likelihoods or evidence ratios (reporting both
would be redundant)."


The top model (when sorted by Delta AIC) always has a model likelihood of 1. Statistically speaking, what does a model-likelihood of 1 mean? Someone may think it means "We're 100% certain this is the correct model." but I know that's not right.


Correct -- look back on page 32 (footnote) -- the following (copied from that page) should explain the distinctions.

You may have noticed a column in the results browser labeled ‘model likelihood’. In fact, this column does not represent
the ML of the model (and is therefore not the value use to calculate the AIC), but rather is the AIC (or BIC) weight for the
model of interest divided by the AIC (or BIC) weight of the best model in the browser. This value is the strength of evidence
of this model relative to other models in the set of models considered, and is the reciprocal of the evidence ratio (discussed
elsewhere). You can add a column showing −2 ln(L) in the results browser but changing the appropriate MARK preference:
File | Preferences.
cooch
 
Posts: 1654
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Re: Reporting AIC weights and model likelihoods

Postby brp » Mon Oct 11, 2010 1:42 pm

Thanks! (Note, after I set the preference I had to close and re-open the file to have the ML appear in the results browser. Is there an easier way?)

I don't think reporting the model likelihoods will benefit (i.e. provide much insight to) the average report reader (non-statistical, non-MARK user). I think reporting the AICc weights only would suffice because evidence ratios can be easily calculated from the AICc weights.

Can anyone give me a convincing argument to the contrary?

Thanks a lot,
Becky
brp
 
Posts: 26
Joined: Tue Sep 29, 2009 12:54 pm
Location: Bozeman, MT

Re: Reporting AIC weights and model likelihoods

Postby cooch » Mon Oct 11, 2010 5:29 pm

brp wrote:Thanks! (Note, after I set the preference I had to close and re-open the file to have the ML appear in the results browser. Is there an easier way?)


No.

I don't think reporting the model likelihoods will benefit (i.e. provide much insight to) the average report reader (non-statistical, non-MARK user). I think reporting the AICc weights only would suffice because evidence ratios can be easily calculated from the AICc weights.

Can anyone give me a convincing argument to the contrary?

Thanks a lot,
Becky


Sure -- the objective is to 'do good science'. If the audience doesn't appreciate certain elements of information, that's their problem. Playing to the lowest common denominator is rarely advisable.
cooch
 
Posts: 1654
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University


Return to analysis help

Who is online

Users browsing this forum: No registered users and 3 guests

cron