Unestimable parameters... which way forward?

[louise.fairless]

Dear all,

I have a-priori considerations that survival and recapture may differ between sex and colony in the mammal I am studying. Therefore I fitted the global model phi(c*s*t)p(c*s*t) to my data where c= colony (3), s=sex(2) and t=time. The overall GOF test in U-care was non significant, therefore this seems an appropriate starting model. However, when fitting the model in Mark, only 305/390 parameters were estimable, probably due to sparse recaptures I imagine.

A similar situation has occured before in the literature(Sendor & Simon, 2003) and in their situation, the authors removed transience from their model and then proceeded to model recapture. Once the most parsimonious model for recapture was found, transience was the put back into the model to estimate survival. I therefore ask if this method is applicable in my situation. I am not using LRT hypothesis testing, but am instead using the information theoretic approach. Instead of keeping phi(c*s*t) constant to model recapture should I remove colony from the survivorship part and keep Phi(s*t) constant whilst I model recapture, then once I have found the best fitting model for recapture, reintroduce the colony effect on survival and estimate survival rates? Or, should I admit that the data is not good enough to support such a complex model and remove the colony factor from the model altogether and start with the global model Phi(s*t)p(s*t)?

Your thoughts on this would be much appreciated.

Best wishes,

Louise

Reference: Sendor & Simon (2003) Population dynamics of the pipistrelle bat: effects of sex, age and winter weather on seasonal survival. Journal of Animal Ecology 72308-320

[dhewitt]

With 85 parameters coming back as not estimable, I don't see how the approach you propose will solve your problems. Not enough data I'd say, and you're probably looking at simpler models than you desire. I'd also suspect that if you look at the components of the GOF test you'll find some problems, even if the overall test seems OK.

[mcmelnychuk]

I have a-priori considerations that survival and recapture may differ between sex and colony

In that case, would model phi(c + s + t) p(c + s + t) suffice?
Or, if you think there might be an interaction between c and s, between c and t, or between s and t, you could consider (for example, for phi) models phi(c + s + t + c:s), phi(c + s + t + c:t), or phi(c + s + t + s:t), respectively. Any one of these, or even models allowing for multiple 2-way interactions, would cut down on the total parameters compared to your global model with the 3-way interaction and all 2-way interactions.

the authors removed transience from their model and then proceeded to model recapture. Once the most parsimonious model for recapture was found, transience was the put back into the model to estimate survival.

Others can weigh in, but it's my understanding that if you want to go for a 2-step approach (first comparing p sub-models, then comparing Phi sub-models) that it's best to maintain as general a sub-model for Phi as possible in the first step. The second steps then involves comparison of the general sub-model for Phi with simpler models rather than more complex models.

Cheers,
Mike

[louise.fairless]

With 85 parameters coming back as not estimable, I don't see how the approach you propose will solve your problems. Not enough data I'd say, and you're probably looking at simpler models than you desire. I'd also suspect that if you look at the components of the GOF test you'll find some problems, even if the overall test seems OK.

I considered this approach because in the aforementioned literature example, nearly 30% of the parameters (62/87) in the global model were not estimable and as I had a similar problem with around 20% of data, I thought it may be a feasible approach. Furthermore, nearly all the parameters in the model phi(s*t)p(c*s*t) are estimable (256/262), so I considered that keeping phi at (s*t) would support the data enough to model recapture. I also take on board Mike's comment (below) which was a concern when trying to justify this method.

Others can weigh in, but it's my understanding that if you want to go for a 2-step approach (first comparing p sub-models, then comparing Phi sub-models) that it's best to maintain as general a sub-model for Phi as possible in the first step. The second steps then involves comparison of the general sub-model for Phi with simpler models rather than more complex models.

However I do understand that the non-estimable parameters highlighted in the global model shows that there is some lack of fit in my data, whether it be due to sparsity or something else, and therefore this approach is perhaps just not good enough. I will therefore look for any problems with the data not initially highlighted in the GOF testing, and also take on board Mike's comments of alternative, appropriate general models.

Thanks for your help so far, it is much appreciated, I will battle on!

Best wishes,

Louise

[dhewitt]

I might have misunderstood what you were planning to do. My thinking was that you were going to model pieces separately and then put them back together (I think I have this right). My concern was that once they were back together you'd still be trying to estimate too much for the data. But if there is considerable reduction in the complexity in one or both pieces, then of course when they go back together there will be less to estimate and perhaps all will go OK. It will depend on the complexity reduction in each piece. Sorry for the confusion if I caused any.

Also, I agree with Mike's suggestion. This might make your approach tricky if the trouble is in Phi, which it appears to be since most of the problem parameters go away when you reduce structure on Phi.

Bummer perhaps, but battle on!

[Doherty]

Gary White, Ken Burnham, and I presented a paper at the last EURING meeting focused on comparing model building and selection strategies such as you are discussing (e.g., p-first, phi-first, all combinations). We conducted simulations in a CJS context. We found that the strategies we investigated had little effect on parameter estimator bias and precision and that model averaging did improve bias and precision slightly. However in terms of variable selection (cumulative AIC weights) the strategies differed, and the all combinations strategy was favored. We view the study as an starting point and more work certainly needs to be done. We did not address issues with datasets that are so sparse that parameters of interest can not be estimated, but the paper may give you some insight into the effects of following different modeling strategies.

The proofs are in and the paper should appear shortly in the Journal of Ornithology (I notice many Euring papers are appearing there in the Online First mode). Or I can send a copy offline to someone that may be interested.

Paul Doherty

[cooch]

Gary White, Ken Burnham, and I presented a paper at the last EURING meeting focused on comparing model building and selection strategies such as you are discussing (e.g., p-first, phi-first, all combinations). We conducted simulations in a CJS context. We found that the strategies we investigated had little effect on parameter estimator bias and precision and that model averaging did improve bias and precision slightly. However in terms of variable selection (cumulative AIC weights) the strategies differed, and the all combinations strategy was favored. We view the study as an starting point and more work certainly needs to be done. We did not address issues with datasets that are so sparse that parameters of interest can not be estimated, but the paper may give you some insight into the effects of following different modeling strategies.

The proofs are in and the paper should appear shortly in the Journal of Ornithology (I notice many Euring papers are appearing there in the Online First mode). Or I can send a copy offline to someone that may be interested.

Paul Doherty

Part of the back-story to Paul's paper (which he refers to) can be found here:

viewtopic.php?f=34&t=393&p=885

The other issue that seems to need stating here is that for models with multiple levels of interactions, you need to think hard about whether the interactions are biologically plausible. If not, then might be little need/motivation to include them. The basic rule of thumb is the more interaction terms you have, the worse your general fit is going to be, since many of said interactions will be very poorly estimated.

[louise.fairless]

Gary White, Ken Burnham, and I presented a paper at the last EURING meeting focused on comparing model building and selection strategies such as you are discussing (e.g., p-first, phi-first, all combinations). We conducted simulations in a CJS context. We found that the strategies we investigated had little effect on parameter estimator bias and precision and that model averaging did improve bias and precision slightly. However in terms of variable selection (cumulative AIC weights) the strategies differed, and the all combinations strategy was favored. We view the study as an starting point and more work certainly needs to be done. We did not address issues with datasets that are so sparse that parameters of interest can not be estimated, but the paper may give you some insight into the effects of following different modeling strategies.

The proofs are in and the paper should appear shortly in the Journal of Ornithology (I notice many Euring papers are appearing there in the Online First mode). Or I can send a copy offline to someone that may be interested.

Paul Doherty

Hi Paul,

Thank you very much for your advice, I would be very much interested in reading your paper. My email address in laf106@soton.ac.uk if it is at all possible for you to send it to me?

Part of the back-story to Paul's paper (which he refers to) can be found here:

viewtopic.php?f=34&t=393&p=885

The other issue that seems to need stating here is that for models with multiple levels of interactions, you need to think hard about whether the interactions are biologically plausible. If not, then might be little need/motivation to include them. The basic rule of thumb is the more interaction terms you have, the worse your general fit is going to be, since many of said interactions will be very poorly estimated.

Thank you for that useful forum link. I agree that thorough considerations of the interactions should be made before developing a global model. Although to my knowledge additive models or models with fewer interactions cannot be tested in U-CARE, the GOF testing options implemented in MARK overcome this if you had a-priori reason to consider a less parametized additive model as your global model as opposed to a fully interactive one.

Thank you for all your discussion and advice.

Louise