model averaging phi p constrained models

[fpc]

My goal is to estimate survival over 5 time periods (in this case river segments). Occassionally those phis near the last time period/reach are highly variable when using the fully parameterized model Phi(time).p(time). This is likely due to relatively low sample sizes. But the premise is one that could apply to any estimation I might do. The model in question does not contain covariates.

Would it be appropriate to use a set of models constraining phis and ps-- essentially reducing the number of parameters setting phi 1 equal to phi 2, or phi 1 to phi 3 etc... and similarly for ps. Then running all combinations of these models. And include the fully time varying model. The goal being to model average the model set to arrive at estimates of all model parameters. And ultimately a model averaged estimate of survival over the entire 5 time periods.

Thanks,

JM

[mcmelnychuk]

JM,

Others may have a different view, but to me that approach sounds a bit odd, unless you have a sound hypothesis why Phi 1 & Phi 2 are more similar than other segments, or Phi 1 & Phi 3 etc. I think if you ran all possible combinations as you suggest, you'll find one or a few that fit the data well, but they'll likely overfit the data; a future dataset might not favor the same model(s).

It sounds like you might have tagged fish migrating past detection stations in a river (or past re-capture locations). If that's the case, you could consider distance-based models, constraining Phi estimates to be a function of the distance of each river segment. There are a couple ways of doing that. First, you could use the distance of each segment as a group covariate, and run model Phi(dist), p(station). You'll get 5 separate p coefficients, an overall intercept coefficient for Phi, and a slope coefficient for the Phi vs. distance relationship (from these the p and Phi estimates are reconstituted, and not constrained to be equal to one another). The other approach (which allows direct estimation of per-km survival rate, I think) is to use the 'Unequal time intervals' option in MARK. Input the (unequal) segment distances in place of time intervals.

Good luck,
Mike

[fpc]

Mike,

I appreciate your comments. Having run survival estimates in these reaches in the past, survival over any segment can vary considerably and for any given reach species combination it is often possible for any combination of reaches to have similar survivals. My goal is to estimate the product of those five reach survivals to get an overall estimate. I'm not so concerned in the analysis about individual reach/time period survival as estimating the entire five reaches with accuracy and precision. From experience I know that constrianing phi equal in all five reaches won't be the best model, but it appears that a fully time varying model may not be supported by the data in many cases.

Others have estimated reach survivals using distance as a covariate for example...and yes I could scale the estimates by distance but ultimately I want to compare overall survival (product of 5 reaches) between groups (time of year), years, ocean indices and ultimately adult return rate. The number of parameters and the combination of in-river and adult return data preclude using the Mark for the entire analysis.

JM

[mcmelnychuk]

It sounds like you're looking for a model intermediate between the fully-pooled Phi(.),p(.) and the fully time-varying Phi(t),p(t) (also, you mention you have several groups, so that general model might be Phi(g * t),p(g * t)). Such intermediate models could involve:
--a model with segment distance constraints on Phi
--a model(s) with groups or species pooled for p, but kept separate for Phi
--a model(s) where p is constrained to be a function of some environmental covariate (flow? river level? direct measures of underwater noise? of if you're working with PIT tags at dams, is the proportion of fish going into the bypass system known independently?).
Consider them all if they're plausible, then let the data arbitrate between them in terms of trade-offs of accuracy and precision.

I've found fairly strong effects of river flow on detection probabilities of acoustic-tagged smolts in the Fraser River, so I'd suggest exploring that if you have environmental covariate data that might affect p. Contact me off-list if you'd like an example.

Last, I wouldn't write off the possibility of tackling large models with MARK. I've run (through RMark) models with nearly 30 smolt release groups that are released in different geographic locations and have some receiver stations in common but others distinct along migration routes. Smolt groups passed between 2-12 detection stations, which resulted in hundreds of parameters. I used models where p was constrained to be common among release groups when it made sense to do so, and separate when that made sense, with separate Phi for each release group & segment. It took a while to run, but it might be preferable to run things in one overall analysis rather than break it up into portions.

[fpc]

I'm asking something a little different here. I realize that identifying the model set is a very critical step in model averaging or multi-model comparisons when hypotheses are being developed and each model represents a competing hypothesis or subset of hypotheses.

But if I begin with the premise that I'm only interested in estimating survival at the end of the fifth time period (in this case) at this stage of my analysis. As such I want the product of the five survival estimates as the ultimate answer. No covariates are considered at this point.

Could I simply use model averaging of some combination of constrained models, in combination with fully time varying model phi(t),p(t) to find the most parsimonious estimates of phis and ps. Here I'm assuming that the phis may be quite similar in any time period (based on experience) and that ps can also be similar from period to period, so that the candidate set of models is not easily identified as a particular subset of constrained models. Would model averaging in that case yield the most unbiased estimate of the overall survival (product of five estimates) given the data set?

Is there a danger of a spurious result when I consider a large group of constrained models given that the product of the five time periods and not individual estimates is really the result I'm looking for. Will this approach yield the most accurate estimate?

[dhewitt]

A question that seems relevant:

- Do you have enough data such that models other than Phi(.) [how appropriate!] are typically supported? If not, why not a dot model estimate used for each river reach?

- Is model selection uncertainty typically large, small, etc.?

The bigger issues you're bringing up are good ones, but perhaps they don't matter as much here.

[sbonner]

Hi JM,

I want to go back to your original question because I'm a little confused. You said that you are interested in estimating survival over all 5 segments of the river (time periods) and that the survival probabilities in the last time period were highly variable when you run a fully time dependent model. My concern is that you can't estimate the survival probability in the last time period with a fully time dependent model because the last survival parameter and last capture probability are confounded. Without further constraints, only their product can be estimated and the individual parameter estimates can't be interpreted. Could this be the problem?

Essentially, if you want to estimate survival over the entire study then you need to add some constraint that allows you to estimate the final capture and survival probabilities. Standard tactics are to assume that survival/capture are the same for the final two segments/locations or to include a covariate -- like the segment length. Without constraints, you can only estimate survival up to and including the second last segment.

Hope that I haven't pointed out the obvious, but I wanted to be sure that this wasn't the problem before making things more complicated.

Simon

[dhewitt]

I believe they have detections of fish beyond the last reach for which they want to estimate survival. I think, although I could be wrong here, that the numbers of tagged individuals is reduced more and more as you go downriver, and the detections beyond the last reach of interest do not generate high re-encounter probs. Thus, the estimation gets worse as you go downriver. I wonder if some of the variability in those later (farther downriver) estimates is just poor precision rather than true variability?

[fpc]

dhewitt is correct.

Sorry I used a bit of shorthand describing the data to be analyzed. Five reaches the phis and ps are estimable. In the sixth reach the phi and p are confounded. That section of river is below the reach of interest for my analysis.

While estimation in and of itself may seem a bit uninteresting, survival estimates are tied to performance standards for the river and therefore unbiased estimates are important.

My thought was that I could use a combination of constrained models as well as full model-- model average them and then recalculate the reach survivals based on those models. Reach survivals are similar enough at different periods in different reaches that based on experience constraining phis in all pairwise combinations although seemingly arbitrary, makes biological sense as well. Will the model average approach improve my ability to estimate with less bias than using the fully parameterized model--which is what we have done historically? Is there a danger of spurious models in this sort of analysis getting a high weight and therefore biasing the overall model averaged estimate?

JM

[sbonner]

JM,

Thanks for clarifying this for me -- I'm sorry to have brought up the red herring.

I agree with Mike that running all possible combinations of models with pairs of survival probabilities set equal is not a good approach. Even with only 5 reaches that's a large set of models, and you're likely to run into some spurious results. I think it's better to try and explain the variation through covariates if you can.

Best,

Simon