Huge CIs - bug?

[lhabib]

I am getting some very strange CIs. With the parameter below, the SE is tiny yet my CIs are huge. Does this have something to do with the fact that it's so close to 1 and is therefore not estimating well? If I calculate the 95% CIs by hand I get 0.85 and 1.0.


Estimate SE LCI UCI
3:Psi 0.9999731 0.1844144E-03 0.0510131 1.0000000

Thank you.

[lhabib]

Sorry about the formatting on that, this should be more clear:

3:Psi
Estimate= 0.9999731
SE= 0.1844144E-03
LCI= 0.0510131
UCI= 1.0000000

[cooch]

I am getting some very strange CIs. With the parameter below, the SE is tiny yet my CIs are huge. Does this have something to do with the fact that it's so close to 1 and is therefore not estimating well? If I calculate the 95% CIs by hand I get 0.85 and 1.0.


Estimate SE LCI UCI
3:Psi 0.9999731 0.1844144E-03 0.0510131 1.0000000

Thank you.

Its not possible to give you a simple answer without knowing more details (especially, type of analysis you're doing, and the size of your data set).

However, in the interim, try re-running the analysis special profile CI's (you click the appropriate box on the right side of the 'run numerical estimation' window).

[lhabib]

Thanks Evan. I'm using "Occupancy Estimation" (16 March 2006 MARK build)

My data sets are typically ~55 rows, with 4 encounters. I'm running 8 models, some of which have the inclusion of a single covariate.

Here's what I get for that same parameter using the Profile Likelihood CI:

Est=0.9999731
SE=0.1844144E-03
LCI=0.8248928
UCI=0.9999997

Seems much more reasonable.

[gwhite]

There is no bug. The confidence intervals on estimates >0 and <1 are computed using a logit transformation to keep the interval boundaries in the feasible range. So, in your example, the psi estimate is converted to ln[psi/(1-psi)] = 10.52336, with SE = SE(psi)/[psi(1-psi)] = 1.84E-4/[0.999973*(1-0.999973)] = 6.855738. Then, upper and lower bounds are computed as 1/(1+exp(-[10.52336+/-1.96*6.855738])) to give exactly the reported values.

This procedure produces asymetric confidence intervals within the 0-1 interval. However, your example demonstrates that the procedure breaks down when the estimate is close to 1 and the SE approaches zero. The zero SE is the result of the estimated variance of a binomial proportion is p(1-p)/n, and as p approaches either 0 or 1, the variance approaches zero.

As Evan suggests, the appropriate method to compute a confidence interval on a parameter at the boundary is to use the profile likelihood procedure.

Gary

[cooch]

There is no bug. The confidence intervals on estimates >0 and <1 are computed using a logit transformation to keep the interval boundaries in the feasible range. So, in your example, the psi estimate is converted to ln[psi/(1-psi)] = 10.52336, with SE = SE(psi)/[psi(1-psi)] = 1.84E-4/[0.999973*(1-0.999973)] = 6.855738. Then, upper and lower bounds are computed as 1/(1+exp(-[10.52336+/-1.96*6.855738])) to give exactly the reported values.

This procedure produces asymetric confidence intervals within the 0-1 interval. However, your example demonstrates that the procedure breaks down when the estimate is close to 1 and the SE approaches zero. The zero SE is the result of the estimated variance of a binomial proportion is p(1-p)/n, and as p approaches either 0 or 1, the variance approaches zero.

The reconstitution that Gary refers to is described in detail in Chapter 7 (the linear models chapter) - see the -sidebar- starting on p. 26 of Chapter 7. It provides a fully worked example of what Gary describes.

As Evan suggests, the appropriate method to compute a confidence interval on a parameter at the boundary is to use the profile likelihood procedure.

The inevitable question that often arises is 'Fine, then why isn't the profile likelihood procedure the default?'. Answer - run a few, see how long they take to compute, then ask yourself the question again.

Given your question, it is probably worth suggesting you fully read Chapter 7 (not just the -sidebar- noted above).

[lhabib]

Fantastic, thanks very much!