Nest Survival Delta Method

[bacollier]

All:
Question I have been struggling with for a few days: how to use the delta method, with model averaged values for ragged telemetry data using the nest survival approach in MARK. Yes, I have read Appendix B.

TIA for any advice, Bret

Scenario:
I evaluated DSR of females for breeding season; best model (w_i=0.32) was 3 period (first 28 days, next 51, last 37) with days tied to reproductive phenology. Model averaged estimates (over the candidate model set) for each period of 0.9992 (0.00038), 0.9986 (0.00041), and 0.9990 (0.00038) respectively.

Thus, for these data, period (breeding season survival) would be

(0.9992^ 28 ) (0.9986^51)(0.9990^37)=0.877325

I have been unable to determine or locate an example or anything even close discussing how to estimate CI's for a point estimate like this given that the VarCovar matrix is based on a single model but the point estimates are model averaged? I assume that there is some sort of weighting of the variances relative to the model averaging and the number of days in each period that will have to occur? I am attaching the .txt and .dbf output, I know that the var/covar values are below the diagonal in the real with the correlations on top, but I thought I would stick both in just in case.

From .dbf output.

PAR1 PAR2 PAR3 PAR4 PAR5 PAR6
0.0000002000 0.0000000000 0.0000000000 0.0000000000 0.0000000000 0.0000000000
0.0000000000 0.0000002000 0.0000000000 0.0000000000 0.0000000000 0.0000000000
0.0000000000 0.0000000000 0.0000002000 0.0000000000 0.0000000000 0.0000000000
0.0000000000 0.0000000000 0.0000000000 0.0000011000 -0.0000001000 0.0000000000
0.0000000000 0.0000000000 0.0000000000 -0.0000001000 0.0000004000 0.0000000000
0.0000000000 0.0000000000 0.0000000000 0.0000000000 0.0000000000 0.0000012000



RGWT DSR (2001-2007) Temporal Analysis
Real Parameter Estimates Variances and Covariances
Model 3--(J=A, J=A, J=A) Different by Region (6 Params-Watch Order)

Variance-Covariance matrix of estimates on diagonal and below,
Correlation matrix of estimates above diagonal.
| 1 2 3 4 5 6
----+------------------------------------------------------------------------
|
1 | 0.0 -0.03885 0.00019 0.0 0.0 0.0
|
2 | 0.0 0.0 -0.00477 0.0 0.0 0.0
|
3 | 0.0 0.0 0.0 0.0 0.0 0.0
|
4 | 0.0 0.0 0.0 0.0 -0.08002 0.00216
|
5 | 0.0 0.0 0.0 0.0 0.0 -0.02695
|
6 | 0.0 0.0 0.0 0.0 0.0 0.0

[cooch]

All:
Question I have been struggling with for a few days: how to use the delta method, with model averaged values for ragged telemetry data using the nest survival approach in MARK. Yes, I have read Appendix B.

<stuff snipped>

Appendix B is a good starting point for deriving the seasonal survival values (and their SE), but, as you've noticed, it doesn't deal explicitly with how to derive values averaged over a bunch of models. I have to decide if this is something I should add, or if in fact it should be fairly obvious what to do if you understand model selection uncertainty, and how to derive the SE and CI for model averaged values - which is discussed earlier in 'the book' (section 6.4.1 in Chapter 6, in fact).

In a nutshell, you simply

1. derive the seasonal estimates for each model, and their respective SE

2. you come up with an estimate of the model average seasonal survival (simple weighted arithmetic mean, weight by normalized AIC weights), and the unconditional SE for the seasonal values averaged over models (as described in section 6.4.1).

3. as noted in Chapter 6, the derivation of the 95% CI is not quite as straightforward as 1.96SE, but its generally not too difficult.

[bacollier]

Thanks for the lead on 6, I had not thought to look outside of Appendix B, blinders must have been on.

Bret

[cooch]

Thanks for the lead on 6, I had not thought to look outside of Appendix B, blinders must have been on.

Bret

Actually, points out the limitation of a sequentially sequenced 'manual', where material in chapter (i) is presented somewhat conditional on the reader having read/assimilated material in all chapters <(i). This is not always reasonable, but somewhat unavoidable, since repeating some conceptual bits in each chapter where they might be relevant would massively increase the length of the book (which is already pretty hefty - Edition 7 is likely to -> 1000 pages, when all is said and done).

[jlaake]

If you are using RMark check out the function deltamethod.special which uses the function deltamethod in the package msm. As a follow-on to his frst RMark script on nest survival models, Jay Rotella is working on another to demonstrate how to do an example of what you are trying to do. It could also be done with output from MARK as long as you create the necessary objects. With the functions above and model.average you should be able to put this together if you are using RMark.

--jeff

[bacollier]

If you are using RMark check out the function deltamethod.special which uses the function deltamethod in the package msm. As a follow-on to his frst RMark script on nest survival models, Jay Rotella is working on another to demonstrate how to do an example of what you are trying to do. It could also be done with output from MARK as long as you create the necessary objects. With the functions above and model.average you should be able to put this together if you are using RMark.

--jeff

Hey Jeff,
I did this one in MARK, but I will port it over to RMARK over break and see if/how I can get it to work. Thanks for the heads up on deltamethod. I look forward to seeing what Jay produces, let me know if you need a another test dataset.

Bret

[jlaake]

You needn't port over to RMark. Simply get the model-averaged estimates and the model-average v-c matrix from MARK. You can then assign them into a vector and matrix in R and use deltamethod.special. Make sure to install and load the package msm first.
If you run into problems, contact me off list. We can post a summary of what worked.

[cooch]

You needn't port over to RMark. Simply get the model-averaged estimates and the model-average v-c matrix from MARK. You can then assign them into a vector and matrix in R and use deltamethod.special. Make sure to install and load the package msm first.
If you run into problems, contact me off list. We can post a summary of what worked.

Absolutely - great time-saver. But, if you want to *understand* what is happening, I strongly suggest you do it manually first, then rely on 'software' to handle the heavy lifting. After all, if you rely on software, you implicitly assume its correct. And testing against my preferred null that software is always wrong requires deriving the alternative hypothesis - which requires doing some things by hand (at least until you have evidence with which to update your prior belief the software is wrong).

Of course, most of the time the software is correct - but not always. For example, MARK reports the estimated number of parameters, but that is determined by the interaction of structure, and data. It is the users responsibility to know how many parameters MARK should be reporting. So, if you rely on what MARK (the software) reports, you'd be wrong in some instances (where the interaction of weird model structure and - typically - sparse data - conspire to lead MARK to reporting the wrong number of parameters).

Moral: it is often unwise to let the 'convenience' of software outweigh the importance of understanding what its doing. You should understand first, use convenience second (IMO).

[bacollier]

You needn't port over to RMark. Simply get the model-averaged estimates and the model-average v-c matrix from MARK. You can then assign them into a vector and matrix in R and use deltamethod.special. Make sure to install and load the package msm first.
If you run into problems, contact me off list. We can post a summary of what worked.

Absolutely - great time-saver. But, if you want to *understand* what is happening, I strongly suggest you do it manually first, then rely on 'software' to handle the heavy lifting. After all, if you rely on software, you implicitly assume its correct. And testing against my preferred null that software is always wrong requires deriving the alternative hypothesis - which requires doing some things by hand (at least until you have evidence with which to update your prior belief the software is wrong).

Of course, most of the time the software is correct - but not always. For example, MARK reports the estimated number of parameters, but that is determined by the interaction of structure, and data. It is the users responsibility to know how many parameters MARK should be reporting. So, if you rely on what MARK (the software) reports, you'd be wrong in some instances (where the interaction of weird model structure and - typically - sparse data - conspire to lead MARK to reporting the wrong number of parameters).

Moral: it is often unwise to let the 'convenience' of software outweigh the importance of understanding what its doing. You should understand first, use convenience second (IMO).

Evan and Jeff:
First, thanks for all your insights. I am comfortable in MARK and getting there in RMARK so I plan to use Evan's suggestion to do it manually first because I want to be sure I understand how it works, port it to RMARK so I can make sure I can get the same output from RMARK (such to teach myself something new in RMARK as I have not done a nest survival problem there yet), and then use the shortcut working with Jeff (offlist). I will document how I did it each step and make a summary available if anyone wants it as I cannot be the first, nor last person to be interested in how this is done, but obviously I was the first to not be able to figure it out

[bacollier]

Evan
Finally got back to this topic: currently working through my data by hand following the MARKBOOK to create a general example of how this is done by hand for everyone, trying to locate the model-specific estimates of variance for the parameter (pg 6-20: est(var(theta_i|M_i)) says they are used internally by MARK for the calculations but not given in output, any idea how to dig them out?

Bret