Robust Design GOF

[bass23]

I'm currently working on a data set in the Robust Design framework. Upon building the global model, I realized that the residuals are mostly above 0 and fewer are below 0, and thus not randomly distributed. After reading the prgm MARK book, this suggests that either I have insufficient data or the model structure does not agree with the data. I tried remedying this problem by changing c-hat, but this did not help much. Is there a method or procedure that would aid in identifying the problem with my global model or data set? Any suggestions would be much appreciated.

[cooch]

I'm currently working on a data set in the Robust Design framework. Upon building the global model, I realized that the residuals are mostly above 0 and fewer are below 0, and thus not randomly distributed. After reading the prgm MARK book, this suggests that either I have insufficient data or the model structure does not agree with the data. I tried remedying this problem by changing c-hat, but this did not help much. Is there a method or procedure that would aid in identifying the problem with my global model or data set? Any suggestions would be much appreciated.

Modifying c-hat doesn't change the lack of fit, it merely adjusts the computed statistics to account for it. So, changing c-hat from 1.0 won't change a thing about how the residuals are distributed - since the structure of the model generating the residuals (given the data) hasn't changed.

As far as identifying the reasons for lack of fit - generally - this is an 'open question'. For live encounter and dead recovery studies, there are several approaches which work pretty well. For things like the RD, I don't believe there are truly any useful diagnostics, short of trial and error with different model structures. For many of the models in MARK (RD included), your left with 'adjusting' for lack of fit (estimating a c-ht adjustment), without really being able to identify why there is lack of fit in the first place.

Note also that methods in MARK for estimating c-hat rely on the assumption that the lack of fit is due to non-independence (i.e., extra binomial noise), and not structural issues. If the reason for lack of fit is structural, and not extra-nomial noise, then estimating and applying c-hat is 'sketchy' - the saving grace is that the worst case scenario in doing this will be an analysis that is more conservative than perhaps need be.

[bass23]

Thanks for responding to my question, but I wanted to follow up with a few other questions if you don't mind. Which methods would allow me to estimate the best c-hat for the data at hand with the RD? Once I've estimated this value, how then do I test if the data fit the global model?

[cooch]

Thanks for responding to my question, but I wanted to follow up with a few other questions if you don't mind. Which methods would allow me to estimate the best c-hat for the data at hand with the RD? Once I've estimated this value, how then do I test if the data fit the global model?

I'm posting the following for Bill Kendall, who is temporarily unable to post things directly:

In terms of an omnibus GOF test, you could run your global model (or the most general one that you can fit) in MSSURVIVRD (www.mbr-pwrc.usgs.gov/software). This produces a Pearson chi-square test after pooling small frequency cells. GOF for RD is an area of development that needs attention.

[bass23]

If small adjustments of c-hat (up to 2) are made and very little changes in model selection and weights are observed (compared to a c-hat value of 1), does this mean that a lack of fit of the global model is due to a structural problem rather than overdispersion, or not necessarily?

[cooch]

If small adjustments of c-hat (up to 2) are made and very little changes in model selection and weights are observed (compared to a c-hat value of 1), does this mean that a lack of fit of the global model is due to a structural problem rather than overdispersion, or not necessarily?

Doesn't mean either, actually - has more to do with the hierarchical structure of your model set.

The positive spin, though, is that if tweaking c-hat 1.0 -> 2.0 doesn't change much, then your inference over that particular set of models is probably relatively robust.

[bass23]

I understand that when I adjusted c-hat that resulted in relatively little changes in model selection, this means my inference is somewhat robust. But, this hinges on my models adequately fitting the models I've selected, correct? I'm still left with my residual deviance plot not appearing very random, does this necessarily mean I for sure do not have a good fit of the global model or do I still need to use a GOF test, to make for certain this is the case? If so, is there a manual or help that I could use to figure out how to run my global model that is currently in MARK in MSSURVIVRD as suggested by Bill Kendall? I appreciate all the help and suggestions thus far.