Strange c-hats for occupancy model

[Summer Burdick]

I am doing a site occupancy analysis with a data set having two capture occasions, 12 groups and five covariates. My groups are defined by a year and substrate type (i.e. 2004 gravel or 2005 sand). A line from my .inp file looks like this:

/* 1 */ 00 1 0 0 0 0 0 0 0 0 0 0 0 0 2.25 3.33 1 0 ;

My most general model is p(group) Psi(group). When I use a .inp file with covariates, my observed c-hat for this model is 2.60. Not great but still less than 3 as recommended on page 5-34 of the most recent version of the MARK manual. When I look at my deviance residuals, they are nearly all outside of the lower confidence bound.

When I try to run a median c-hat test I get the error message “every one of your simulated values generated a c-hat value greater than your observed c-hat value, logistic regression cannot be performed”

Then I tried to run a Bootstrap GOF and I get estimates of c-hat in the range of 38.98 to 42.75!

I also input an .inp file containing no covariates. An example line from the inp looks like this:

/* 1 */ 00 1 0 0 0 0 0 0 0 0 0 0 0 ;

When I run the same general model as before {p(group) Psi(group)}, I get a observed c-hat of 43.00. However, using the .inp file without covariates I get residuals symmetrically distributed about 0 and well within the confidence bounds.

I get the same wacky results when I run more constrained models.

What am I doing wrong? I am pretty sure my model doesn’t fit supper well, but is it time to throw in the hat or rethink my model structure?

[cooch]

I am doing a site occupancy analysis
...
What am I doing wrong? I am pretty sure my model doesn’t fit supper well, but is it time to throw in the hat or rethink my model structure?

Actually, as far as I can tell, you're not doing anything wrong - my guess is that there are some 'issues' wrt GOF testing, occupancy models, and MARK. I'll see if I can try a few things over the week-end.

In the meantime, I'd try the following. Run your models using the default c-hat of 1.0. Then, once you've run all the models, see how sensitive your model rankings are to small, incremental increases in c-hat (say, by manually adjusting it from 1.0, 1.2, 1.4...2.4). If you're lucky, your best (most parsimonious) models won't change much as you change c-hat.

Stay tuned...

[cooch]

Read things more closely this time...

I am doing a site occupancy analysis with a data set having two capture occasions, 12 groups and five covariates. My groups are defined by a year and substrate type (i.e. 2004 gravel or 2005 sand). A line from my .inp file looks like this:

/* 1 */ 00 1 0 0 0 0 0 0 0 0 0 0 0 0 2.25 3.33 1 0 ;

My most general model is p(group) Psi(group). When I use a .inp file with covariates, my observed c-hat for this model is 2.60. Not great but still less than 3 as recommended on page 5-34 of the most recent version of the MARK manual. When I look at my deviance residuals, they are nearly all outside of the lower confidence bound.

When you look at deviance residuals for a model with individual covariates, the deviance residual plot is not meaningful (you can prov this for yourself by taking any of the example files from Chapter 12 that have individual covariates - run them, and look at the deviance residual plots - they're all 'strange'). It has to do with the way the likelihood is constructed when you have individual covariates in the .INP full - even if you don't actually use the covariates in the analysis, the likelihood is still constructed the same way.

When I try to run a median c-hat test I get the error message “every one of your simulated values generated a c-hat value greater than your observed c-hat value, logistic regression cannot be performed”

I'm puzzled, since the median c-hat is not available for models with individual covariates.

I also input an .inp file containing no covariates. An example line from the inp looks like this:

/* 1 */ 00 1 0 0 0 0 0 0 0 0 0 0 0 ;

When I run the same general model as before {p(group) Psi(group)}, I get a observed c-hat of 43.00. However, using the .inp file without covariates I get residuals symmetrically distributed about 0 and well within the confidence bounds.

OK, so without the individual covariates, the deviance residual plots are meaninful.

And, also, remember that the observed c-hat by itself doesn't say much - its how the observed compares to the distribution of expected c-hats given your data that is more relevant. Try running median c-hat on the model without individual covariates, and see what happens.

Basically, if the model without individual covariates fits, then the model with covariates will fit also. This is discussed briefly at the end of chapter 12.

[Summer Burdick]

Sorry for the confusion about which model I was testing. I didn’t run the median c-hat test on a model that had covariates. I ran a median c-hat test on a model without covariates but I used an INP file with covariates in it. I understand now that I need to use an INP file without covariates to estimate c-hat.

I tried running a median c-hat test using the INP file with no covariates and got the same message as before, “every one of your simulated values generated a c-hat value greater than your observed c-hat value, logistic regression cannot be performed”.

I also tried inputting a range of c-hat values to see how sensitive my AIC ranking was to changes in c-hat. I found that for values of c-hat between 1.25 and 4.00 the order of the top six models (as ranked using AIC) did not change. For c-hat>6 more constrained models were favored.

Following a suggestion by Mat Alldredge (NCSU) I attempted to simulate data by setting c=1. For these simulations I selected the same general model I examined before {p(group) Psi(group)} using the Results Database button on the True Model tab. First I let MARK fill in the beta values and number of plots using the actual data. Then in a second simulation I increased the number of plots in each group from around 40 to 200. For both simulations I set c=1 on the Simulation Specification tab. In both cases the c-hat from the simulated data was approximately equal to the observed c-hat from the real data (~ 43). I think the c-hat from the simulated data should be approximately equal to 1. Is that true? Should I be doing something different in my simulation attempts? Or, is this large simulated c-hat an indication of model bias? If so how do I adjust for it?

Thanks for your help!

Summer