Bootstrapping the age model in MARK book - page 7-13

[trrobbin]

At the bottom of p. 7-13, the last sentence of the second to last paragraph reads: "While you could use the bootstrap here - no need, since the data are simulated under this model [the general model Phi(a2-t/t)p(t)] , and thus c-hat = 1.0"

Well, just for learning purposes I ran the bootstrap GOF on the general model, Phi(a2-t/t)p(t), and after comparing where the model deviance (128.5) falls into the simulated model deviances, I got a p-value of 0.01 meaning that the model does not fit the data well, correct? Something seems amiss.

Or am I confused about what that is supposed to mean?

The estimated c-hat using deviance values = 1.33 and using c-hat values = 1.35.

[cooch]

At the bottom of p. 7-13, the last sentence of the second to last paragraph reads: "While you could use the bootstrap here - no need, since the data are simulated under this model [the general model Phi(a2-t/t)p(t)] , and thus c-hat = 1.0"

Well, just for learning purposes I ran the bootstrap GOF on the general model, Phi(a2-t/t)p(t), and after comparing where the model deviance (128.5) falls into the simulated model deviances, I got a p-value of 0.01 meaning that the model does not fit the data well, correct? Something seems amiss.

Or am I confused about what that is supposed to mean?

The estimated c-hat using deviance values = 1.33 and using c-hat values = 1.35.

You need to go back and re-read pp. 38-40 in Chapter 5. The data were simulated under the noted model - true c is 1.0. c-hat (which is an estimate) is likely to differ from 1.0 for any given sample (each sample is a single realization of the underlying stochastic process specified by the model). In other words, even if true c=1, sample c-hat might be quite different from 1 (see histogram and accompanying text on p. 39 of Chapter 5). Generally, since inference is made from the sample, you'd want to use sample c-hat, but the point of the exercise in chapter 7 is how to build the models. The assumption is that if you'd read Chapter 5, and we're doing this for real, you'd know what to do.

[trrobbin]

You need to go back and re-read pp. 38-40 in Chapter 5. The data were simulated under the noted model - true c is 1.0. c-hat (which is an estimate) is likely to differ from 1.0 for any given sample (each sample is a single realization of the underlying stochastic process specified by the model). In other words, even if true c=1, sample c-hat might be quite different from 1 (see histogram and accompanying text on p. 39 of Chapter 5). Generally, since inference is made from the sample, you'd want to use sample c-hat, but the point of the exercise in chapter 7 is how to build the models. The assumption is that if you'd read Chapter 5, and we're doing this for real, you'd know what to do.

I can understand that the book was trying to show us what to do in reality. Actually, that is what I was worried about. I really wasn't worried about the estimate of c (c-hat), so much. One could easily add c-hat to the model. I was more worried about the bootstrap GOF test. If it says that data simulated for a model does not fit the model, than how much should I rely on the bootstrap GOF test for my real data, which is of course not going to be as perfect of a model fit? A p-value of 0.01 is rather significant.

I guess you would just add the c-hat to the model and run a bootstrap GOF again. It just doesn't bode well for the bootstrap GOF test.


Thank you!

[trrobbin]

You need to go back and re-read pp. 38-40 in Chapter 5. The data were simulated under the noted model - true c is 1.0. c-hat (which is an estimate) is likely to differ from 1.0 for any given sample (each sample is a single realization of the underlying stochastic process specified by the model). In other words, even if true c=1, sample c-hat might be quite different from 1 (see histogram and accompanying text on p. 39 of Chapter 5). Generally, since inference is made from the sample, you'd want to use sample c-hat, but the point of the exercise in chapter 7 is how to build the models. The assumption is that if you'd read Chapter 5, and we're doing this for real, you'd know what to do.

I can understand that the book was trying to show us what to do in reality. Actually, that is what I was worried about. I really wasn't worried about the estimate of c (c-hat), so much. One could easily add c-hat to the model. I was more worried about the bootstrap GOF test. If it says that data simulated for a model does not fit the model, than how much should I rely on the bootstrap GOF test for my real data, which is of course not going to be as perfect of a model fit? A p-value of 0.01 is rather significant.

I guess you would just add the c-hat to the model and run a bootstrap GOF again. It just doesn't bode well for the bootstrap GOF test.


Thank you!

[cooch]

You need to go back and re-read pp. 38-40 in Chapter 5. The data were simulated under the noted model - true c is 1.0. c-hat (which is an estimate) is likely to differ from 1.0 for any given sample (each sample is a single realization of the underlying stochastic process specified by the model). In other words, even if true c=1, sample c-hat might be quite different from 1 (see histogram and accompanying text on p. 39 of Chapter 5). Generally, since inference is made from the sample, you'd want to use sample c-hat, but the point of the exercise in chapter 7 is how to build the models. The assumption is that if you'd read Chapter 5, and we're doing this for real, you'd know what to do.

I can understand that the book was trying to show us what to do in reality. Actually, that is what I was worried about. I really wasn't worried about the estimate of c (c-hat), so much. One could easily add c-hat to the model. I was more worried about the bootstrap GOF test. If it says that data simulated for a model does not fit the model, than how much should I rely on the bootstrap GOF test for my real data, which is of course not going to be as perfect of a model fit? A p-value of 0.01 is rather significant.

I guess you would just add the c-hat to the model and run a bootstrap GOF again. It just doesn't bode well for the bootstrap GOF test.


Thank you!

I think you're missing the point. In reality, what you'd do is estimate c-hat for your sample, and apply that. If the bootstrap (or median c-hat, which is more robust - see Chapter 5) says that a particular sample doesn't meet the expectations given that (for simulated data) the general model is known, then this in fact may not be surprising. Again, look at the histogram in Chapter 5 - it shows clearly that over a very large number of replicate simulations, where true c-hat=1.0, the 'estimated' c-ht for a large number of the simulated datasets is <1, and a large number is >1. The fact that the bootstrap or media c-hat reports values that are <>1 is *not* indicative of problems with the bootstrap or median c-hat (which is what you imply). In fact, they're showing you exactly what you would expect. c-hat is an *estimate*, and for some samples, the estimated c-hat might be quite different than the true c-hat. Generally, this is a value of the effective sample size.

So, for the example presented in Chapter 7 (to which you originally referred), the sample size in age.inp is relatively small, and even though true c-hat is 1.0, the estimated c-hat is in fact 1.25 (or so) - you get more or less the same estimate regardless of whether or not you use the bootstrap, median c-hat, or RELEASE (which can be tweaked to give you c-hat for that particular model).

In fact, I should probably note this in Chapter 7 - in reality, for that example, you'd want to use the c-hat=1.25 adjustment, but I didn't bother noting that simply because the point of the exercise in Chapter 7 is on how to build the models.