model-averaging estimates of individual covariates

[zugunruhe1]

Hello, I am doing nest survival analysis and was wondering how to get model-averaged estimates and unconditional SE's for individual predictors (i.e. percent grass cover around nest). I see that there is an example of how to calculate the unconditional SE for the entire model set on p 4-48 and an example of how to get survival estimates for specific values of individual covariates in Ch 11 but that isn't helpful to what I am trying to do- any advice would be much appreciated.

Thank you!

[cooch]

Hello, I am doing nest survival analysis and was wondering how to get model-averaged estimates and unconditional SE's for individual predictors (i.e. percent grass cover around nest). I see that there is an example of how to calculate the unconditional SE for the entire model set on p 4-48 and an example of how to get survival estimates for specific values of individual covariates in Ch 11 but that isn't helpful to what I am trying to do- any advice would be much appreciated.

Thank you!

I think you might be reading an older version of both chapters. Using the current, online versions of Chapter 6, have a read of the -sidebar- starting on page 49 of Chapter 6, and then, once you've realized its a lot of work to do it by hand, have a careful read of section 11.8 in chapter 11. Unless I'm missing something in what you're after, this is exactly what you want: model averaged S on the vertical axis, plotted against a range of the covariate (grass cover) on the horizontal axis. As per section 11.8, you can do this directly from the 'model average | individual covariates' menu option.

[zugunruhe1]

Thank you very much! It sounds like maybe what you're describing would still be for a range of covariate values, however. Maybe posting an example of exactly what I'm trying to get will help:

Parameter Estimate Unconditional SE 95% CI Odds ratio
Connecting canopy 2.398 0.903 0.628 to 4.168 11.003
Nest cavity depth -0.010 0.006 -0.021 to 0.001 0.990

This is borrowed from a table in a publication presenting model-averaged parameter estimates for models of macaw nest survival. I am wondering how a single estimate for parameters with differing values was obtained.

Thanks once again.

*Because it looks like the formatting got messed up in the posting process, to clarify, in the first line of the table,
parameter = connecting canopy
estimate = 2.398
Unconditional SE = .903
95% CI= .628 to 4.168
Odds ratio=11.003

[cooch]

Thank you very much! It sounds like maybe what you're describing would still be for a range of covariate values, however. Maybe posting an example of exactly what I'm trying to get will help:

Code: Select all

 Parameter          Estimate     Unconditional SE     95% CI            Odds ratio
Connecting canopy   2.398           0.903           0.628 to 4.168       11.003
Nest cavity depth  -0.010           0.006          -0.021 to 0.001        0.990


This is borrowed from a table in a publication presenting model-averaged parameter estimates for models of macaw nest survival. I am wondering how a single estimate for parameters with differing values was obtained.

What they've done (or, 'attempted') is to model average the beta parameters for various parameters, rather than the reconstituted real parameters. In general, this is *not* a good idea (which, in fact, is why MARK doesn't let you model average betas directly as a 'menu option').

Have a look at

viewtopic.php?f=21&t=2923

and

viewtopic.php?f=1&t=996


The usual aim of model averaging betas is to get an average estimate of the 'effect size'. For reasons discussed in the preceding threads, this isn't as easy as it sounds. One approach is to simply calculate the model averaged estimates of reals, and then use any differences between levels of the reals as a measure of the effect size (and then us, say, the Delta method, to derive the SE of this difference).

[zugunruhe1]

Thank you once again for an informative reply!

I read all the material you suggested, and I think I’d have a problem generating model-averaged values over a range of the covariate because my models use differing PIM structures (i.e. some age-dependent, some not, some treat years as different, some do not). The chapter identifies different PIM structures as a problem but does not seem to go into how to address this problem.

I also wanted to clarify what you mean by:

One approach is to simply calculate the model averaged estimates of reals, and then use any differences between levels of the reals as a measure of the effect size (and then us, say, the Delta method, to derive the SE of this difference).

I believe you’re saying that to get an idea of the effect of grass cover, for example, I could use the difference between the model averaged estimate of nest survival for a very low grass cover value and a very high value as an indication of effect size. However, my differing PIM structures would still present a problem in getting these estimates.

[cooch]

Thank you once again for an informative reply!

I read all the material you suggested, and I think I’d have a problem generating model-averaged values over a range of the covariate because my models use differing PIM structures (i.e. some age-dependent, some not, some treat years as different, some do not). The chapter identifies different PIM structures as a problem but does not seem to go into how to address this problem.

You're over-interpreting the 'warning'. If your most general model is (say) a2: t/t (two age classes, full time-dependence for both age classes), then you (i) set up the PIM structure to reflect that, (ii) build the DM corresponding to that structure, and then (iii) build all of your remaining models by manipulating the DM from this general model (e.g, a model with no age classes is constructed simply by deleted the age offset, and the interaction of 'age and time'). As long as the underlying PIM structure is the same over your models, you're fine. If it isn't, then the solution is to rebuild your model set based on a common underlying PIM structure (it really isn't that hard -- consult chapter 6 and 7 for the various steps/trick for building the age models).

I also wanted to clarify what you mean by:

One approach is to simply calculate the model averaged estimates of reals, and then use any differences between levels of the reals as a measure of the effect size (and then us, say, the Delta method, to derive the SE of this difference).

I believe you’re saying that to get an idea of the effect of grass cover, for example, I could use the difference between the model averaged estimate of nest survival for a very low grass cover value and a very high value as an indication of effect size. However, my differing PIM structures would still present a problem in getting these estimates.

No, but estimating an 'overall effect size' for a factor (say, sex), is complicated by the fact that the magnitude of the 'sex effect can vary a lot over intervals. For example, if you had significant support for (say) an interaction of sex and time, then trying to talk about a 'model-averaged' effect of sex doesn't make sense in the first place, since the effect of sex varies over time (this is what an 'interaction' of effects means in the first place).

I understand the attraction with being able to make one sweeping statement about thee 'effect of something'. While you can talk about the overall support for that effect, quantifying it can be somewhat more complicated. But, model-averaging over individual covariates provides an approach -- you can even use this approach for classification factors, simply by coding them as individual covariates (this is also discussed in chapter 11).

[zugunruhe1]

Hi again, I read over your reply several times and I understand your point on effect magnitude depending on the interval, especially with an interaction. However, I'm mainly trying to figure out what you mean when you refer to "levels" of the reals.
Thank you!

[cooch]

Hi again, I read over your reply several times and I understand your point on effect magnitude depending on the interval, especially with an interaction. However, I'm mainly trying to figure out what you mean when you refer to "levels" of the reals.
Thank you!

Simple. Say you have some real covariate (size), and two sexes you're interested in comparing. For a given value of the covariate, you estimate the real parameter (say, survival) for each sex. The effect size is simply the arithmetic difference between the male and female real estimates. SE for the effect size approximated using the Delta method (Appendix B).

[zugunruhe1]

Hi, I'm sorry to bug you even more all the replies you've posted to this thread. So, having read through the thread a few times again and read about the Delta method and through section 11.8, I’m still having trouble understanding how to apply the ideas you provided. Your suggestion to get model-averaged effect sizes of covariates on survival was

“…to simply calculate the model averaged estimates of reals, and then use any differences between levels of the reals as a measure of the effect size (and then us, say, the Delta method, to derive the SE of this difference)”

and that a level would be something binary, like sex. However, all of the variables in my top models that I would want to get an effect size for are *continuous* numerical variables, either I'm missing something (definitely possible!) or that presents a problem.

(also, for the record, the reason I'm beating this into the ground despite all the caveats is that my advisor is very adamant about wanting overall effect sizes!)

Thank you.

[cooch]

and that a level would be something binary, like sex. However, all of the variables in my top models that I would want to get an effect size for are *continuous* numerical variables, either I'm missing something (definitely possible!) or that presents a problem.

For a single, continuous covariate, single model, the slope is the effect size. Related to log-odds. For model averaging, simply output the model averaged values for the covariate, take the difference at two model-averaged reals that differ by one unit of the covariate, calculate the difference (which is the slop), and go from there. Binary contrast of points on a line. Same difference.

(also, for the record, the reason I'm beating this into the ground despite all the caveats is that my advisor is very adamant about wanting overall effect sizes!)

This would be the advisor who apparently isn't able to help you with this.