GOF test error message

[Tracy]

I am trying to estimate survival for a small marked fish population (n=350) using both a CJS model and a Barker
model where I have relatively few recaptures (e.g., the
data is too sparse to support a time-dependent Barker model or any similarly parameterized model). Recaptures for the Barker are both live/dead and ongoing/discrete. When I run a RELEASE GOF test for the CJS model, it says there is insufficient data to run several of the contingency tests. When I try to use a median c-hat or a bootstrap GOF test on any of my models, either using CJS or Barker, I get the following error:

Visual Fortran run-time error
Forrtl: severe (161): Program Exception – array bounds exceeded
Image PC Routine Line Source
MARK.EXE 004D53AE ESTMAT 860 estmat.for
MARK.EXE 0059EF6C MARK 279 mark.for
MARK.EXE 00668789 Unknown Unknown
MARK.EXE 0065172B Unknown Unknown
Kernel32.dll 7C80B713 Unknown Unknown

First, what does this error mean? Is this happening because my data
are simply too sparse to test for goodness of fit? If so, is there any way around it? My deviance residual plot looks abysmal (most points
occur below the lower dashed line), but I think this is because in the
majority of encounter histories, an individual was marked and never
seen again.

Finally, without running a GOF test or estimating variance, can I still trust the survival estimates from my top models?

Thank you,
Tracy

[chellman]

Hi Tracy - I don't have the answer for you, but I'm running into the same problem right now. I'm running the data using a CJS model and our recapture rates are also fairly low. Any ideas out there?

[cooch]

Hi Tracy - I don't have the answer for you, but I'm running into the same problem right now. I'm running the data using a CJS model and our recapture rates are also fairly low. Any ideas out there?

Quick thoughts:

1/ if p<0.15, then there are going to be significant limits to your inference. Period. There are some *big names* (who I will let self-identify if they wish) who will suggest that for p<0.15, you might as well give up - or, perhaps somewhat more optimistically, not expect much of interest. Further, quoting from Chapter 5 (GOF chapter of the MARK book):

...if you are sure your model structure is correct, and despite you’re finer efforts, your c-hat is  >3, or if your model rankings change radically with even small changes in c-hat, then you might just have to accept you don’t have adequate data to do the analysis at all (or, perhaps in a more positive tone, that there are real limits to the inferences you can draw from your data). Unfortunately, your ’time, effort, and expense’ are not reasonable justifications for pushing forward with an analysis if the data aren’t up to it. Remember the basic credo. . .‘garbage in...garbage out’.

Same applies (and is not unrelated to) the issue(s) created by low encounter probabilities.

2/ if p>0.15 (i.e., in the realm of 'acceptable), but you're still having problems, then it is more than likely that your general model (for which you are trying to assess GOF) is overly parametrized *for your data*. Similar to point /1, some will say that if you don't have sufficient data to test GOF for a time-dependent general model, then doing GOF for a less-general model is not useful, since if you're general model is already a reduced parameter model, then the model set is unlikely to contain other even more reduced models that will be of much interest.

The whole issue of 'how much data do you need?', and 'what is the minimum encounter probability to be useful?' has been considered in great depth. Short of reading a fairly copious literature, you're advised to use simulation to estimate 'power' needed for your study (see Appendix A for simulations in MARK).

[Tracy]

chellman: I received that error message because I tried to run the GOF test using an input file that included covariates. So even if the model you run the GOF test on does not have covariates, your input file cannot include covariates. That may be your problem too?

Of course, everything Evan replied with is true. With low estimates of p, you're very limited in what kind of models you can try to fit, and consequently how much biological inference you can make. However, I have found that the RELEASE GOF and median c-hat tests will run, even with sparse data. The RELEASE test will tell you if you have insufficient data for each of its tests. Although this won't help you with your present data set, it could help you assess how to improve your p in the future.

Also, looking at the graph of your deviance residuals can be helpful in determining if your data is not distributed normally. Again, make sure when you look at the graph that it is based on your most global model from an input file WITHOUT covariates.

Hope that helps,
Tracy