Model averaging with group covariates

[jtebbenkamp]

Hello,
I am working on a survival analysis and have successfully built and run the necessary models using RMark, however, am having issues with model averaging using both the RMark package and with the program MARK interface. Due to how the data were collected I'm using a nest survival model broken into monthly intervals allowing me to produce estimates of monthly survival rates. All of the variables included in my models are categorical. The variables include year (2007, 2008, 2009, 2010, 2011), season (breeding, summer/fall, and winter), reproductive status (successfully nested or did not successfully nest), age class (adult or yearling), and study area (3 different study sites).
Year and season appear to be two of the most important variables, but since there is not a single model that received overwhelming support (model weight for the top model is 0.23) it seems model averaging would be appropriate. I would like to get model averaged results for the average monthly survival rate during each year and then separately the monthly survival rate during each season The problem that I am running into is that rather than obtaining a model averaged survival rate during each year or during each season I am getting model averaged results for each different combination of covariates (e.g. 2007 - breeding season - successful nest - study area 1, 2007 - breeding season - unsuccessful nest - study area 1, 2007 - breeding season - successful nest - study area 2, 2007 - breeding season - unsuccessful nest - study area 2… And the list goes on).
I understand how to use model averaging with individual covariates such as body mass or wing chord but am falling short when trying to model average with group covariates. Is there a way to produce model averaged results with group covariates that does not return a model averaged survival estimate for each combination of groups but only for the variables of interest (e.g. year or season)? Any advice or suggestions would be greatly appreciated.
Thank you,
Joel

[cooch]

The default in MARK is to average over all orthogonal factors in the underlying (general) parameter structure (time x group etc). For example, suppose your general model is phi(sex*time)p(sex*time). MARK will generate model averaged values for each time interval, and for each sex. What you want is something analogous to getting averages for each interval, pooling between sexes, or for each sex, pooling over time.

There are a couple of ways you can approach this.

1\ by hand. Averaged values are relatively easy -- they're functions of AIC weights. SE for things is a bit more complicated -- Delta method. Some of this might be straightforward using RMark, but its hard to say.

2\ a more sophisticated approach is to use the MCMC capabilities of MARK. You could estimate mu and sigma for any hyperdistribution you want (e.g., pooling over sex for each time interval...). MCMC is covered in Appendix E (still in draft, but the basics are there).

[jlaake]

In RMark I'd suggest computing your averages for each model and their std errors with one of the deltamethod functions and then model average those with the general model.average.list. It is typically easiest to compute whatever statistic you want for each model and then model average those quantities rather than model average the estimates and then compute the statistics (eg mean) on those values.

But I'm wondering whether you have thought through what you are trying to estimate particularly when you are averaging across a variable like successful/unsuccessful. A simple average assumes a 50% split if you are making inference to the population mean survival. It is similar to a numeric covariate prediction. For the population you would compute the survival at each wing length and then weight by the fraction in the population that have that wing length. That can be quite different than the survival at the mean wing length depending on the distribution of wing lengths. A simple possibly non-realistic example might be high survival for short and long-winged birds and near zero survival for medium winged birds. If there are very few birds with medium wing length, the average survival for the population will be high but the survival at the average wing length could be near zero.

regards --jeff

[jtebbenkamp]

Hello,
Thank you both for your suggestions. After looking further into the different options suggested, I think the best way forward for me would likely be computing the averages and standard errors for each model and then model average those results.
The major problem I am running into now deals with the point that Jeff brought up regarding the fact that if given 2 groups a simple average assumes a 50-50 split. I was thinking a potential way around this might be to use a weighted average by computing the averages for each model weighted by the number of individuals represented by each group (e.g. each combination of variables). Would this be acceptable to do? If so, that is good because it gets me one step closer, but if not is there a different way to go about this? Even if it is okay I still have one more potential issue…
If I were to choose to model average with only the models less than 2 deltaAIC from the top model, I would still be dealing with 4 different group covariates. Based on the different combinations of these 4 variables this leaves me with 75 different groups, each with its own estimate for survival. The problem I perceive is that some of these parameter estimates are based on only 1 or 2 individuals in a group leaving me with somewhat iffy parameter estimates and large CIs. Is this an issue or would these issues get washed out due to the fact that a weighted average would be more heavily influenced by the estimates from the groups that are represented by a larger, more adequate, number of individuals?
Thanks again,
Joel

[jlaake]

You should either be thinking about the MCMC approach Evan suggested or a variance components approach. The only variable that is problematic with what I suggested is reproductive success because it is not fixed like the others: season, year, area, age-class. Presumably you want a survival rate for each age-class in each season that is averaged over year and area treating them as random effects. Using a sample frequency is fine as long as there is no sampling bias (eg successful more likely to be sampled). Also note that the weights are random quantities and should contribute to the variance.

jeff

[jtebbenkamp]

Thank you again for your for your advice. Based on your suggestions and after reading over the appendices again I think I am now on the right track.
Thanks,
Joel

[egc]

Thank you again for your for your advice. Based on your suggestions and after reading over the appendices again I think I am now on the right track.
Thanks,
Joel

In order for Appendix E to make much sense, you need to work through Appendix D, first.