Stepwise optimization of model parms, particularly in RD

[Miguel]

My usual approach to analyses in MARK is to optimize the fit of p (and c in robust models) while constraining all other parameters as a single rate, i.e., (.). Using the model of p with the most support, the other parameters are then optimized one at a time.

Using my approach with the Robust model example RD_Complex.inp in the MARK book, the ‘correct’ p model p=c(good/bad years) received little support relative to ‘wrong’ p models. The 'correct' p model only become apparent after the other parameters were optimized. If this had been a real analysis where the answer was not known, my approach would have led to misleading results, This result suggests that a fishing expedition would be necessary to find the most appropriate parametrization of p, which I'm sure is not the desired approach.

Is my usual approach to model fitting not appropriate for robust models?

Your thoughts are greatly appreciated,

Thanks,
Miguel

[cooch]

My usual approach to analyses in MARK is to optimize the fit of p (and c in robust models) while constraining all other parameters as a single rate, i.e., (.). Using the model of p with the most support, the other parameters are then optimized one at a time.

Not recommended in general as a robust approach. Paul Doherty and Gary have recently published a paper on this.

http://www.springerlink.com/content/y6l ... lltext.pdf

Using my approach with the Robust model example RD_Complex.inp in the MARK book, the ‘correct’ p model p=c(good/bad years) received little support relative to ‘wrong’ p models. The 'correct' p model only become apparent after the other parameters were optimized. If this had been a real analysis where the answer was not known, my approach would have led to misleading results, This result suggests that a fishing expedition would be necessary to find the most appropriate parametrization of p, which I'm sure is not the desired approach.

Is my usual approach to model fitting not appropriate for robust models?

Your thoughts are greatly appreciated,

Thanks,
Miguel

As per my original comment -- the step-wise approach is *not* recommended in general. You've just demonstrated why. This approach is usually taken as a 'short-cut' to handling what could be a very large model set. I would submit that taking a short-cut (using stepwise approaches) to 'save time' is not particularly 'scientific'. A big model set will take time to construct. My quick comment to that observation is .... so what?

[Miguel]

Thank you so much for your reply and for the link. I'm looking forward to reading Paul and Gary's paper.

I'm helping out a grad student with her analysis and it is interesting how people working independently with the same data can arrive at different best fit models. This is an area of mark-recapture analysis that needs a more consistent methodology and or a greater awareness of one.

Thanks again,
Miguel

[cooch]

Thank you so much for your reply and for the link. I'm looking forward to reading Paul and Gary's paper.

I'm helping out a grad student with her analysis and it is interesting how people working independently with the same data can arrive at different best fit models.

which is why we do model averaging, which takes into account model specification uncertainty (which is essentially the basis of the difference you refer to).

This is an area of mark-recapture analysis that needs a more consistent methodology and or a greater awareness of one.

Miguel

I would submit the problem is the latter of the two.

[Miguel]

Yes, model averaging helps parameter estimates reflect model uncertainty. However, unless I am mistaken, separate analyses can produce similar averaged parameter estimates using best fit models that are sufficiently different to lead to different conclusions about what influences the estimated population rates.

It would be interesting to post on phidot simulated data (with multiple groups, time specificity, etc) and compare both the parameter estimates and the interpretation of model fit among different folks/volunteers. Food for thought.

Thanks for your comments Even.
Miguel