Delta Method and 95% CIs

[Tony G]

I've taken up the Delta Method challenge to calculate the variance of back-transformed estimates in order to estimate 95% CIs (in my case the influence of water temperature on detection probability for frogs). To better understand the workings of the Delta Method I've used Example 4 (Appendix B-19) to re-create the values (using excel). I successfully estimated the variance by calculating the partial derivatives, then multiplied the vector of partial derivatives with the variance-covariance matrix to estimate the variance and then taken the square root of this to get the SE. This felt pretty good as I'd been looking at this chapter for years with some trepidation . (I even managed to detect a very minor typo on page B-21 - a zero is missing on the variance estimate after the matrix calculation).

I went an calculated variance on my own data and then estimated 95%CI using the steps listed below:

step 1 - calculate 95% CI logits - b1+(b2*stdz covariate) +- 1.96*sqrt(variance)
step 2 - back-transform 95% CI logits - exp(logit)/(1+exp(logit)

Logits B1 -0.970978 (se 0.206664), B2 -1.221872 (se 0.236884), Stdz covariate value - -3.04651446159902

V-C B1 0.042710 0.020334
B2 0.020334 0.056114

However, I ended up with suspiciously small 95%CI values that I don't have much faith in.

I'd appreciate any insights into where I may have erred on this please.

Regards
Tony

[JeffHostetler]

Tony,

I tried your example and got a SE on the logit scale of about 0.663, and a confidence interval on the real scale of about 0.810 - 0.983. If you got a smaller CI and SE, it might be because you calculated the variance for the real scale, which is what they did for Example 4 in Appendix B. To calculate CI the way you present, it should be the logit scale variance/SE that should be used. Fortunately, this is simpler to calculate. The partial derivative of b1+(b2*stdz covariate) to b1 is 1 and to b2 is the value of the covariate. So the variance of the formula via the delta method is var(b1) + (value of stdz covariate)^2 * var(b2) + 2*(value of stdz covariate)*cov(b1,b2).

Or if you get lazy or want to check your math, the deltamethod function in the msm package in R works pretty well.

Hope this helps!

Jeff

[cooch]

Please also have a read of the -sidebar- in Chapter 6 -- pp. 26-28. In particular, the part on p. 28 about how the 95% CI is derived when reconstituting estimates on the real scale:

The standard approach to calculating 95% confidence limits for some parameter θ is θ ± (1.96 × SE). However, to guarantee that the calculated 95% CI is [0, 1] bounded for parameters (like φ) that are [0, 1] bounded,MARK first calculates the 95% CI on the logit scale, before back-transforming to the real probability scale. However, because the logit transform is not linear, the reconstituted 95% CI will not be symmetrical around the parameter estimate, especially for parameters estimated near the [0, 1] boundaries.

[Tony G]

Hi Jeff,

Your advice was right on the money - thankyou. Using the variance formula you provided I was able to recreate the example and came up with the same answer as you did, and then tested it using the deltamethod in the msm package in R. I was confused between working with the real and logit variances due to relying on the worked examples. I had gone through the information in Chapter 6 and Appendix B but didn't comprehend how to actually do it using the logits.

I'm off to buy a phi-dot t-shirt now .

Thanks.

Tony

[cooch]

Hi Jeff,

Your advice was right on the money - thankyou. Using the variance formula you provided I was able to recreate the example and came up with the same answer as you did, and then tested it using the deltamethod in the msm package in R. I was confused between working with the real and logit variances due to relying on the worked examples. I had gone through the information in Chapter 6 and Appendix B but didn't comprehend how to actually do it using the logits.

I'm off to buy a phi-dot t-shirt now .

Thanks.

Tony

Indeed -- Jeff provided the mechanics to derive the 95% CI on the real scale. In looking at TFM, I did already have a demonstration of the basic point for example (3) (sidebar starting on p. 16), but that only involved a single variable - which was trivially easy (since you could simply read the SE for the variable from the MARK output for the beta estimates). So, I just added another sidebar (for example 4) for the situation similar to what you were facing, where you had a multivariable function to work with. New version of the appendix (posted a little while ago), sidebar beginning on p. 19). Since example (5) is just a variation on example (4), I don't replicate the basic point a third time.

One 'gotcha' to be aware of - the expression you differentiate wrt to each Beta needs to be written additively. In example (4), the actual linear model is evaluated to have a positive beta(1) term, and a negative beta(2) terms. But, the equation you differentiate is beta(1)+beta(2)(covariate).