Overfitted models.

questions concerning analysis/theory using program MARK

Overfitted models.

Postby sergiodel90 » Wed Mar 24, 2021 4:16 am

Hi everyone!

I am doing a research and I send my manuscript and one of the reviewer said about my table results Mark Programm:

"The AICc values were relatively high, and the number of parameters were too big. This model would cause the overfitting from the data set."

My best model has 5665 AICc and 18 np. Deviance 1735.

What should I do for check my overffited?
sergiodel90
 
Posts: 4
Joined: Thu Jun 13, 2019 5:49 am

Re: Overfitted models.

Postby simone77 » Wed Mar 24, 2021 1:04 pm

The AICc values make sense in relative terms (not in absolute terms). I would say the same for the deviance. The difference in AICc between two models says how much relative support has one model with respect to the other. The difference in deviance between two models says how much deviance is explained by one model with respect to the other (in case the two models differ in just one "covariate", it is how much deviance that covariate explains). If the number of parameters is too high or not depends on the degrees of freedom you have (i.e. it has to do with the sample size).

A first starting point to understand if your model is overfitted is checking the standard error of your parameters' estimates. How large are the 95% CI of those 18 estimated parameters? Another relevant issue is the extrinsic non-identifiability of your model that is the parameters that the model cannot estimate due to "problems" with the data (relative to the model you have set up). Regarding the latter, you may want to have a look at 4-71 (Addendum – counting parameters) in the Program MARK – "A Gentle Introduction". You may want to have a look at a recent post on parameters identifiability here.
simone77
 
Posts: 197
Joined: Mon Aug 10, 2009 2:52 pm


Return to analysis help

Who is online

Users browsing this forum: No registered users and 9 guests

cron