GOF for closed/removal study?

[jbaum]

Is there a way to test GOF or estimate c-hat for removal data (c=0) from closed populations other than the chi-square in Capture's limited generalized removal estimator?

Thanks

Jeremy

[bmitchel]

MARK does not allow bootstrapped GOF estimation for closed population models or models with individual covariates. Does anyone know why?

Is it possible to implement a GOF simulation for closed population models using the following method?
1) Run your model to estimate closed population parameters
2) In Excel, generate a bootstrap data set based on the parameter estimates that has the same number of individuals captured and marked as the original data set.
3) Run this data set through MARK (or a spreadsheet that can fit the model and calculate the deviance).
4) Repeat 2 and 3 for 100 iterations.
5) Use the output deviances as discussed in Evan Cooch's book, in Chapter 5, to evaluate fit and estimate c-hat.

Similarly, couldn't this approach be used for models with individual covariates (with the modification that covariate values for each individual in simulated data sets are chosen with replacement from the measured covariate values in the original data set)?

This seems pretty straightforward, so is there a theoretical quagmire that I am not aware of?

[gwhite]

Brian:
The problem is that you have to condition on the number of animals captured to do your bootstrap. This conditioning means that you underestimate the variance in the original data. This is the mistake in the Minta - Mangel estimator for the mark-resight approach. See the following paper for a more thorough explanation and demonstration of the problem:

White, G. C. 1993. Evaluation of radio tagging marking and sighting estimators of population size using Monte Carlo simulations. Pages 91-103 in J.-D. Lebreton and P.M. North, eds. Marked individuals in the study of bird populations. Birkhäuser Verlag, Basel, Switzerland.

Gary

[bmitchel]

At the risk of extending this thread beyond its useful life, I'd be interested in people's thoughts on the following...

I looked at the reference Gary cited, and at the 1989 Minta and Mangel paper that proposes their estimator. I clearly erred in suggesting that a GOF/c-hat estimation simulation be based only on the number of marked animals. As Gary wrote in 1993, a better approach is to base each simulated population on the estimated population size (and should also take into account the variability in the estimated population size... for example by using a population size generated from a normal distribution having the mean and standard deviation produced by the initial data analysis). This probably still does not fully model the variance of the initial data, and would therefore result in a GOF and c-hat that are biased low.

My thought is that I would rather have a c-hat that is pretty good and that I know is biased low than no estimate of c-hat at all (the current situation for closed-capture models). This would allow me to examine the sensitivity of my results using a much narrower range of potential c-hat values.

Similarly, I think this approach could be extended to calculating bootstrap GOF and c-hat for models with individual covariates. Again, I acknowledge that the bootstrap would likely result in estimates of GOF and c-hat that are biased low, but isn't a biased estimate better than no estimate at all?

Thanks,

Brian Mitchell
Postdoctoral Research Associate
University of Vermont

[cooch]

...
Again, I acknowledge that the bootstrap would likely result in estimates of GOF and c-hat that are biased low, but isn't a biased estimate better than no estimate at all?

A biased estimator is not, in and of itself, problematic. An estimator of unknown bias (magnitude, direction) is a problem. By and large, both are unknown (in many cases) for estimators of c (i.e., c-hat) - Gary has done a fair bit of work exploring bias with bootstrapped estimates for some models, but in general, the bootstrap (and more recently, median c-hat), are (as noted in various places) works in progress.

So, a biased estimate is not necessarily better than no estimate. Take GOF for MS models. There is the contingency approach in MS-SURVIV. There is the median c-hat, and program U-CARE. They can almost be guaranteed not to give the same estimates of c-hat, for a given set of data. So, you could, of course fight the philosphical fight of whether you're more comfortable making a type I or type II error. Most folks seem to prefer to be conservative, pick the larger c-hat, and live wth the reality that the best models will be reduced parameter models relative to what they might have selected with another (lower) c-hat.

This basic idea is also behind the other, cruder approach that has been recommended on occasion, and used on occasion: you manually adjust c-hat over a range - say, starting with 1.0, and increasing in increments of 0.25 over the range 1.0 -> 3.0, and look to see what happens to the rank order of models. If you're living lucky, the ranking of the better models won't change much.

And, finally, its worth remembering that our estimate of c is just that, an estimate (hence, c-hat). If you look at the last few pages of the GOF chapter, you see an example of this - even though the true model might have a c-hat of 1, for any given data set that is a realization of the underlying model structure might yield a c-hatquite different from the true 1.0. So, even if the c-hat estimate is unbiased, its still an estimate.