Individual covariate significance vs. AICc

[cbonen]

Dear all,

We are trying to test for the effect of an individual covariate
(heterozygozity) on survival rate using Mark. Something like:

logit(phi) = beta_0 + beta_1 * covariable

with an overall effect of heterozygozity on all age-classes;

Our model is a *multi-state model*, with 2 different states to account
for trap-dependence problems and *unequal time intervals*. After fitting the model with and without the individual covariable, we end up with the following AICc table, as provided by Mark:

-----------------------------------------------------------------
Model AICc K Deviance
-----------------------------------------------------------------
Model without covariable 4477.1150 46 4383.0190
Model with covariable 4476.4067 47 4380.2185
-----------------------------------------------------------------

Change in number of parameter is ok. However, once investigating beta
estimates, beta_1 corresponding to the individual covariate is:

------------------------------------------------------------
Parameter Beta se Lower Upper
------------------------------------------------------------
...
50: 0.0639 0.0067 0.0507 0.0771
------------------------------------------------------------

which suggests a statistically significant beta_1 and hence, a
significant effect of the individual covariate (after standardization)
on survival since 0 is not included within the 95% confidence
limits.

If barely significant variables may lead to similar AICc values, we do
not expect similar AICc with what appear as a significant
beta_1. Performing a LRT test, chi^2 = 2.800, 1 df, the p-value is
0.0942 which is again in agreement with AICc. However, such results as
given by AICc and the LRT sound, to us, inconsistent with the beta_1
(+/- sd). We would like some light to be shed on this.

Thanks for any thoughts,

Christophe Bonenfant

Université Claude Bernard, Lyon 1

[dhewitt]

The confidence interval "test" and LRT you mention are aimed at something different than what AICc is aimed at. (And, confidence interval coverage ought not to be interpreted as a test in general.) The two sources of information aren't inconsistent, they're just different.

From your table you can see that the covariate absorbed some variation left in the data (reduced the deviance). But, AICc is telling you that that amount of additional variation explained is not much once you account for estimating another parameter. The effect of the covariate isn't nil, it's just not very parsimonious to include it.

In this case, what does the actual magnitude of the beta estimate translate into on a real variable scale? Is it possible that the effect is consistent but simply small (i.e., unimportant)?

[jlaake]

I would look at how the other parameters changed when the new variable was added. My guess would be that the new variable abosorbed the predictive value of other covariates leaving some of them redundant.

In general there is a correspondence between LRT and AIC with nested models and addition of a single parameter and this in turn is roughly related to the p-value of the coefficient. But when you have many different covariates that are possibly correlated, all bets are off!

--jeff