Multi-season Models and GOF

[Bird Counter]

Recently, I saw a thesis in which the author proposed using c-hat values from single-season models to adjust standard errors in model-averaged multi-season model estimates. I had never heard of this. He did not find evidence of overdispersion in his two single-season models so he made no adjustments. It is not clear whether he would have adjusted the SEs of psi1, gamma and eps using the sqrt(c-hat) value he calculated from the single-season models. It also was not discussed how he would have dealt with different estimates of c-hat in the single-season models.

I have assessed GOF in single-season models and have found some evidence of overdispersion (mean c-hat <1.7) in one year. However, I did not use this information when constructing multi-season models. This approach does not seem to be the right way to deal with this issue, but I cannot find other ways to deal with it in the applied literature. I am only talking about situations where overdispersion is not excessive.

Any thoughts from the experts in the field?

[jhines]

At the moment, there is not goodness-of-fit test for multi-season models, so if you're really worried about fit, you could calculate the chi-square gof stat from each single-season model, add them up and divide by the sums of degrees of freedom to get an overall c-hat for the multi-season model. Otherwise, you could simulate data with the most complicated model and compare the likelihoods of the simulated data sets vs the real data. I think if care is taken in sample design and modeling, gof would not be as important since the gof test is usually based on the assumption of large sample sizes with normal distributions (statisticians can feel free to 'flame' me if I'm way off base here).

[Bird Counter]

Thanks much! Your response is very helpful.

[Diego.Pavon]

Hi,

I am having also problems with GOF. I am not quite sure how to do it.
I have 22 seasons (years) and 3 samples per season. The method that you (Jim) suggest here, seems a bit tricky. If I understood correcly, I would have to make 22 single-season projects and then test GOF and sum them up. I have tried to make singe-season models in the same project where I am working now (multi-season models) and it does not seem to work. Maybe I am not doing it properly. And also, Should I do it with mi "best models" (i.e. the 4 models that were ranked at the top within 2 AIC units) or shall I use the more parameterized model (fully t dependent) even though it did not converge?
In the example with the salamanders (MacKenzie & Bailey 2004, assessing the fit on site-occupancy models) they use the model with covariates in stead. I am using 4 covariates to model detection probability. When trying to run single-season models in this project I can't use the 2 covariates because they are site and survey specific. In addition, my best models are p(Seasonal effect + Location). In this sense, I can't run these models in a single-season framework.

Do you have any suggestion on how to assess GOF in this case?

Thank you very much

[jhines]

You need to create each single-season analysis in a separate folder. Presence will get confused if different data and results files are in the same folder.

For GOF, you should use the most general model in your model-set, not necessarily the one with the lowest AIC. It should be the one with the lowest -2*log-likelihood value, though. It sounds like the most general model (time-specific detection) didn't converge due to sparse data, so I'd throw that one out. Then, find the most general model.

You should be able to use the site and survey specific covariates for the single-season analysis. Just subset the covariates in the same way you subset the detection data. For season one, you're only using the first few columns of the detection data, so use the same columns from the survey-specific covariate(s).

If your most general model is p(seasonal effect + location), you could add a more general model, p(seasonal effect * location) to the model-set. That model would translate to p(location) for each season.

Jim

[Diego.Pavon]

Hi Jim,

Thank for the comment. You suggested to run the model p(seasonal effect * location). The thing is that location is (1) categorical variable (location 1 or 0) and coded as site specific covariate. How can make an interaction between the seasonality and the site-specific covariate? And between seasonality and any other covariate (continuous or factor)? I don't know, would it look something like this?
b1 b2 b3 b4 b5 b6 b7
1-1 1 0 0 0.2 0.2 0 0
1-2 1 0 0 0.2 0.2 0 0
1-3 1 0 0 0.2 0.2 0 0
2-1 0 1 0 0.3 0 0.3 0
2-2 0 1 0 0.3 0 0.3 0
2-3 0 1 0 0.3 0 0.3 0
3-1 0 0 1 0.5 0 0 0.5
3-2 0 0 1 0.5 0 0 0.5
3-3 0 0 1 0.5 0 0 0.5

Thank you very much

Diego

[jhines]

To model the interaction of season*covariate, you can use the following design matrix:

Code: Select all

   b1 b2 b3 b4 b5 b6
1-1 1 0 0 0.2  0   0
1-2 1 0 0 0.2  0   0
1-3 1 0 0 0.2  0   0
2-1 1 1 0 0.3 0.3  0
2-2 1 1 0 0.3 0.3  0
2-3 1 1 0 0.3 0.3  0
3-1 1 0 1 0.5  0  0.5
3-2 1 0 1 0.5  0  0.5
3-3 1 0 1 0.5  0  0.5

where column 'b1' is the 'intercept', column 'b2' is the 'effect of season 2', column b3 is the 'effect of season 3', column 'b4' is the effect of your covariate, columns 'b5' and 'b6' are the interaction effects (of season and the covariate).

Another design matrix which produces the exact same estimates of detection is:

Code: Select all

   b1 b2 b3 b4 b5 b6
1-1 1 0 0 0.2  0   0
1-2 1 0 0 0.2  0   0
1-3 1 0 0 0.2  0   0
2-1 0 1 0  0  0.3  0
2-2 0 1 0  0  0.3  0
2-3 0 1 0  0  0.3  0
3-1 0 0 1  0  0   0.5
3-2 0 0 1  0  0   0.5
3-3 0 0 1  0  0   0.5

where column 'b1' is the 'intercept' for season 1, column 'b2' is the 'intercept for season 2', column b3 is the 'intercept for season 3', column 'b4' is the effect of your covariate for season 1, columns 'b5' is the effect of your covariate for season 2, and 'b6' is the effect of your covariate for season 3.