Confusing confidence interval

[jjroper]

Hello all and Happy New Year.

I am getting a consistently strange confidence interval in an analysis. In fact, it always seems to be in the second line...here is the output - note the second line, especially the standard error and the lower confidence interval. Can anyone explain to me why the second (in this case age group) has a lower confidence interval than the rest, even though the standard error is not large and why such assymetry?

Real Function Parameters of {Phi age 5, p constant}
95% Confidence Interval
Parameter Estimate Standard Error Lower Upper
------------------------- -------------- -------------- -------------- --------------
1:Phi 0.9637101 0.0094911 0.9397628 0.9783564
2:Phi 0.9977995 0.0097691 0.0689049 0.9999996
3:Phi 0.9577356 0.0149984 0.9163983 0.9790996
4:Phi 0.9213018 0.0254000 0.8548970 0.9587824
5:Phi 0.8079684 0.0875859 0.5818651 0.9271216
6:p 0.2490161 0.0362261 0.1848921 0.3264721

Thanks,

Jim

[cooch]

Ola Jim --

I am getting a consistently strange confidence interval in an analysis. In fact, it always seems to be in the second line...here is the output - note the second line, especially the standard error and the lower confidence interval. Can anyone explain to me why the second (in this case age group) has a lower confidence interval than the rest, even though the standard error is not large and why such assymetry?

Real Function Parameters of {Phi age 5, p constant}
95% Confidence Interval
Parameter Estimate Standard Error Lower Upper
------------------------- -------------- -------------- -------------- --------------
1:Phi 0.9637101 0.0094911 0.9397628 0.9783564
2:Phi 0.9977995 0.0097691 0.0689049 0.9999996
3:Phi 0.9577356 0.0149984 0.9163983 0.9790996
4:Phi 0.9213018 0.0254000 0.8548970 0.9587824
5:Phi 0.8079684 0.0875859 0.5818651 0.9271216
6:p 0.2490161 0.0362261 0.1848921 0.3264721

Thanks,

Jim

First, parameters that are near either the [01] boundary are poorly estimated (regardless of the link function used, although the sin does better than the logit). This estimation uncertainty is often reflected in strange SE and CI calculations.

Second, instead of the default CI, try using calculations based on a profile likelihood (see last section of chapter 1 in the book). You do this by clicking the 'Profile Likelihood CI' option in the 'Setup Numerical Estimation Run' window.

Let us know what you see after making this change.

[jjroper]

Thanks! I should have remembered that.... Here are the new CIs

Parameter Estimate Standard Error Lower Upper
------------------------- -------------- -------------- -------------- --------------
1:Phi 0.9637101 0.0094911 0.9458773 0.9830377
2:Phi 0.9977995 0.0097691 0.9781609 1.0000000
3:Phi 0.9577356 0.0149984 0.9276133 0.9859659
4:Phi 0.9213018 0.0254000 0.8671314 0.9673461
5:Phi 0.8079684 0.0875859 0.5578813 0.9249155

Now they make sense! I appreciate the help!

Jim

[cooch]

Thanks! I should have remembered that.... Here are the new CIs

Parameter Estimate Standard Error Lower Upper
------------------------- -------------- -------------- -------------- --------------
1:Phi 0.9637101 0.0094911 0.9458773 0.9830377
2:Phi 0.9977995 0.0097691 0.9781609 1.0000000
3:Phi 0.9577356 0.0149984 0.9276133 0.9859659
4:Phi 0.9213018 0.0254000 0.8671314 0.9673461
5:Phi 0.8079684 0.0875859 0.5578813 0.9249155

Now they make sense! I appreciate the help!

Jim

Good -- glad it worked out.

And, in case anyone asks "if profile Ci are 'better', why aren't they calculated by default?', the answer is...because they require significantly more computation than do the standard CI's. This is the same reason that simulated annealing isn't the default numerical optimization routine, and so on...

[dhewitt]

Jim,

You still need to be careful here. Profile likelihood CIs often perform poorly near the boundaries. From the MARK help file:

"In summary, care should be taken in using the profile likelihood capability because of the numerical problems caused with parameters at the boundary. Unfortunately, parameters estimated at the boundary is the primary reason that the profile likelihood interval is desirable."

We run into this all the time. Unless I know there is a pile of data informing the parameter and it is reasonably THAT close to 1.0, I consider it a poor estimate due to data issues. Note that the top end of the CI you were dealing with is still at 1.0 -- which ain't a good sign.

- Dave

[cooch]

Jim,

You still need to be careful here. Profile likelihood CIs often perform poorly near the boundaries. From the MARK help file:

"In summary, care should be taken in using the profile likelihood capability because of the numerical problems caused with parameters at the boundary. Unfortunately, parameters estimated at the boundary is the primary reason that the profile likelihood interval is desirable."

We run into this all the time. Unless I know there is a pile of data informing the parameter and it is reasonably THAT close to 1.0, I consider it a poor estimate due to data issues. Note that the top end of the CI you were dealing with is still at 1.0 -- which ain't a good sign.

- Dave

True, but the profile is still better than the default CI, which can have an upper CI >1 (or <0, in some instances), which clearly makes no sense. The fact the Jim's reported UCI is 1.0 indicates that there isn't enough data to do a particular good job. However, in practice, the LCI from the profile is generally still pretty reasonable, in my experience.

[dhewitt]

I've seen a number of times where the lower CI from the profile CI is just as silly as the default. Be cautious...