Observer experience & Royle & Link's misclassificati

[hf.hwa]

Hi,

I’m interested in accounting for “observer experience” in Royle & Link’s model that allows for both false negative and false positive errors.

Here’s our situation. We used 4 groups of 3 observers each to survey for a forest pest. Two of the groups were comprised entirely of 3 volunteers with no prior experience while the remaining two groups had 1 and 2 experienced observers each, the remaining individuals in these groups being volunteers. Groups were assigned sites at random and individuals within groups visited sites independently.

As a proxy for abundance, we returned to all trees where the pest was detected and counted the number of individuals observed. Because this pest is sessile and because the surveys were completed on the same day (1) heterogeneity in surveys should be entirely a function of observer skill and (2) there will be no change in abundance between surveys. Of course we may not have detected / counted all individuals on our return surveys however.

We considered four models that either allowed for false positives or did not and in which surveys (observers in this case) differed in detection and misclassification probabilities.

We found that: (1) groups with experienced observers detected the pest at a greater proportion of sites, (2) experienced individuals detected smaller populations than volunteers, (3) experienced individuals had a higher probability of detecting populations than volunteers, and (4) surprisingly, experienced individuals ALSO HAD A (MUCH) HIGHER PROBABILITY OF MISIDENTIFYING the target species than volunteers.

This latter finding is easy to explain when we consider the detection histories. The most common detection history was one in which an experienced observer detected populations when two of the volunteers did not – thus in terms of the model, it is more likely that the experienced observer is ‘wrong’ and the two volunteers are correct. Thus the model assigns a high misclassification probability to the experienced observer, resulting in a biased estimate of site occupancy. However, our return surveys to count abundance suggest the opposite, namely that the experienced individual detected a very small population that the two volunteers failed to detect.

Our initially surprising results arise from the fact that observers with different abilities occur in the same group, but these differences are not incorporated in the model. This situation causes great problems for the misidentification models and results in estimates of site occupancy that are extremely biased.

My question is: Is there a way to account for heterogeneity in detection / misclassification probabilities given that they are related to observer experience and abundance? Would a simple covariate (1 = experienced, 0 = volunteer) do it?

Thanks in advance!

[darryl]

With this Royle and Link model, essentially what it's assuming is that there are 2 different types of sites, and the probability of detection is lower at 1 type than the other. Royle and Link called the group of sites with this lower probabiltiy of detection as the misclassified sites, but in reality this could also arise through heterogeneity and have nothing to do with false positives. From your description, it sounds like it's heterogeneity that is causing your problem, not false positives. My believe is that the only way to reliably tease out false positives (when you also have nondetections) is with some additional information (ie training programs or lab ids). I'd caution you to think carefully about the results from this model. That said, yes you could just include an indicator covariate to indicate which survey was conducted by an experienced person.

[hf.hwa]

Hi Darryl - Many thanks for your reply. I greatly appreciate your help on this problem.

I agree that the problem is because of heterogeneity & not from false positives (the pest is unmistakable to the trained eye - there is nothing else like it in the forest). Basically, the experienced observer is detecting a small population that the volunteers miss, but the single experienced observer gets "out voted" by the two volunteers. I'm I interpreting this correctly?

I find my results very interesting and agree that the results should be interpreted with extreme caution. I'm in the process of writing my findings up for publication & I am curious if there are any published examples in which "experience" has been used as a covariate? I have yet to come across any.

Thanks again & best wishes,
Matt

[darryl]

I agree that the problem is because of heterogeneity & not from false positives (the pest is unmistakable to the trained eye - there is nothing else like it in the forest). Basically, the experienced observer is detecting a small population that the volunteers miss, but the single experienced observer gets "out voted" by the two volunteers. I'm I interpreting this correctly?

Sounds like a reasonable interpretation. If that is the case though, then why are you using this model? I'd suggest you just use the standard models.

I am curious if there are any published examples in which "experience" has been used as a covariate? I have yet to come across any.

Can't say for sure, but I expect there would be. It's something I've used myself in analyses for clients pretty often.

[mcmelnychuk]

I am curious if there are any published examples in which "experience" has been used as a covariate? I have yet to come across any.

Hi,

I'm not sure about occupancy models, but see this reference for an example of an experience covariate (i.e., efficiency of individual snorkelers) in a mark-recapture model.

Korman, J., Ahrens, R.N.M., Higgins, P.S., and Walters, C.J. 2002. Effects of observer efficiency, arrival timing, and survey life on estimates of escapement for steelhead trout (oncorhynchus mykiss) derived from repeat mark-recapture experiments. Can J Fish Aquat Sci 59(7): 1116-1131.

cheers, Mike

[hf.hwa]

Sounds like a reasonable interpretation. If that is the case though, then why are you using this model? I'd suggest you just use the standard models.

Good question. When I compared (using AIC) the standard model to the misclassification model, the misclass model has the strongest support (by far).

Thanks again,
Matt

[darryl]

But if it's not a biologically reasonable model, why consider it in the first place?