Prediction Limits

[neilmidlane]

I recently completed a single-season single-species occupancy study using the Custom with Spatial Correlation options in Presence. I had surveyed 41 of 73 sites, and extrapolated my results to produce a map of spatially explicit probability of occupancy, as well as a coefficient of variation for each cell (as per Sunarto et al. 2012 Plos ONE: Tigers need cover: multi-scale occupancy study of the big cat in Sumatran forest and plantation landscapes). My study is in review at a journal and one of the reviewers is insisting that I should present "prediction limits" rather than CVs along with my occupancy probability for each cell to give an indication of the variability in the data. However, I haven't come across this anywhere in the occupancy literature. Is the reviewer correct? If so, can PRESENCE produce prediction limits? If not, can anyone advise on how I would go about calculating them?

Thanks
Neil

[darryl]

PRESENCE does give you confidence intervals for estimated occupancy probabilities, if that's what the reviewer is referring to by 'prediction limits', although usually a prediction limit (or interval) demonstrates the degree of variation for a single realisation of some process while the confidnece interval demonstrates the uncertainty in the expected value of some process. So it depends on exactly what your reviewer means by the terms, and why are they insiting on it over a CV, as to whether PRESENCE already has the required info in the output for you or not.

Cheers
Darryl

[danimal]

I would cite other published examples where a common format is used and request the reviewer to clarify the issue. It is possible (and very probable [depending on the journal]) that the reviewer is not familiar with what it is that you are doing or how to articulate their own concerns.

[neilmidlane]

Thank you for the replies. For more clarity, below is the comment from the reviewer in the first round of reviews and my response:

Reviewer: I suggest that you do not extrapolate your model to areas that you did not survey. If you DO extrapolate, then keep in mind that prediction limits, rather than confidence limits, should properly be your estimate of uncertainty for these projections. (Consider that the CL for the overall mean PAU (Proportion of Area Used) ranges from 0.53 - 0.91.given this, the prediction limits probably range almost 0 -1). If you present spatial representations of PLU, you must also provide maps that reveal the uncertainty about PLU. It is important that the uncertainty is not downplayed when extrapolating from 18 transects to the entire park, particularly in products such as maps that might be 'under-analyzed' by users

Response: We surveyed 41 of 73 grid cells (i.e. 56.2%). By comparison Sunarto et al. (2012) surveyed ±15% of cells in their study area, whilst Linkie et al., (2006) surveyed only 3.8% (200 of 5262) cells in theirs, yet both these authors extrapolated to unsurveyed sites, and presented predictive maps similar to ours. The latter presented no data on variability in their estimates in their map, while the former present representative imagery for coefficient of variation (CV) in each cell.

Based on the approach adopted in these studies and our greater sampling effort we felt justified in extrapolating our results. However, we appreciate the cautionary note when considering users of the map, and we have thus refined our map to graphically present CVs in each cell (new Figure 3) following Sunarto et al. (2012), as these result in a less cluttered representation than trying to represent prediction limits. We add this step to our methods (line 297 in revised MS).

Here is the second response from the reviewer:

Concern about extrapolation of the results to un-surveyed areas still remains, even though the rebuttal cites precedents for such extrapolation. If the extrapolation is retained, then prediction limits must be shown, because confidence limits do not correctly describe the uncertainty of these extrapolations. The concern is not addressed by the statement "we refined our map to graphically present CVs in each cell (new Figure 3) following Sunarto et al. (2012), as these result in a less cluttered representation than trying to represent prediction limits." One can't validly present a CV for a cell that one did not study. Although prediction limits will (correctly) be noisier than these CVs, that is exactly the point.

In light of this and your replies to my original post, could either of you provide some more guidance?

Thanks very much!

Neil

[darryl]

Hi Neil,
This sort of help is possibly pushing the intended usage of the forum, but here goes anyway.

I'm really puzzled by the reviewer's comment about extrapolation, that's exactly the sort of thing you should be doing with your results (whole idea of statistics is do extrapolate from a sample to a population of interest). The only good reason I can think of for why you wouldn't extrapolate is if there's something about the way you selected the cells to survey that means they may not be a good representative sample of the entire area. Perhaps the reviewer has some concern about the methods which is leading them to that comment.

I disagree with the reviewer about being unable to present a CV for a unsurveyed cell, the key is how to calcuate the CV appropriately which is similar to the CI vs PI argument. All this depends on exactly what you're trying to represent with your map. Is it; a) presence/absence of the species in each cell; or b) estimated probability of presence in each cell. The 2 are similar and related, but subtly different.

The former case is a single realisation of a single draw from a Bernoulli process (ie a binary random outcome, eg a coin flip) for each cell. If there was perfect detection and psi.hat was the estimated probability for an unsurveyed cell, the standard deviation would be sd=sqrt(psi.hat*(1-psi.hat)), and you CV for each cell would be sd/psi.hat. With imperfect detection, it's going to be bigger than that, but I don't know the exact right formula in that case (though it could be derived from the delta method if you really want to). For the surveyed cells you'd do something a bit different becuase you know the species is present in some of them (where you detected it at least once) and there's no uncertainty associated with those observations. For those surveyed cells where the species was not detected, you don't know for certain the species is absent (unless overall detection was almost 1), but you could use the conditional psi values from the PRESENCE output (prob. occupied given species never detected there) as your point estimate, call it psi.c.hat, and the sd would be similar to above but using psi.c.hat instead of psi.hat.

In the later case where you're mapping the estimated probabilty of occupancy, that's where I think it's ok to just be using the CI's and CV as it sounds like you've already done. This is because the probability is the underlying parameter that governs the realisation of the Bernoulli process that determines where the species is present or absent. Note that we could know that probability exactly (ie CV=0), but there's still going to be varation in where the species is actually present at a particular time point due to the random Benoulli process.

So if you're trying to map likely presence/absence of the species then I'd agree with the reviewer that your CV's probably aren't large enough, but if you're trying to map probabilities then I'd argue what you're doing is probably ok (but I could be wrong).

Cheers
Darryl

[neilmidlane]

Hi Darryl

Thank you very much for the comprehensive reply, I really appreciate the assistance!

Kind regards
Neil