Mark residual plots

questions concerning analysis/theory using program MARK

Mark residual plots

Postby nlsanto2 » Wed Jul 24, 2013 10:08 am

Hello,
I am a graduate student at Sapienza University of Rome and I for my master thesis I am analyzing mark-recapture data from a striped dolphin population in the gulf of Corinth, Greece using the sub-module POPAN in program MARK 7.1.

I am trying to evaluate the fit of POPAN models to my data using the residual plots.
I found some statements on the Internet of people who say that they think that the residuals plots in MARK are not reliable and that there may be a bug.

What I would like to know is:
are the residuals plots reliable in MARK?
is there any situation in which they should not be used?
is there any literature or manuals (except for the Mark Manual) that I can consult about this topic?

Thank you to anybody who woould answer! :?:
Nina
nlsanto2
 
Posts: 18
Joined: Wed Mar 14, 2012 5:01 am

Re: Mark residual plots

Postby abreton » Wed Jul 24, 2013 6:52 pm

Nina,

Probably you've seen these discussions in the MARK manual ... just in case you haven't check out,

Chapter 5, Pages 34-35
Chapter 7, Pages 31-33
Chapter 7, Page 39 (Section 7.5)
Chapter 11, Pages 46-47 (main text and sidebar)
Chapter 12, Pages 14, 19 (a case where residual plots cannot be used)

I found 'residual' mentioned several times in the appendices as well using a search of the 'whole' mark book.

Some additional info under the mark HELP menu ... 'residual plots'.

andre
abreton
 
Posts: 111
Joined: Tue Apr 25, 2006 8:18 pm
Location: Insight Database Design and Consulting

Re: Mark residual plots

Postby nlsanto2 » Thu Jul 25, 2013 8:57 am

Dear Andre thank you very much for your tips.

I have already searched the manual and the forum (I should have mentioned that, sorry) but I have not found the answer to my questions. Which are essentially:

How are the expected values computed in MARK?
Are the residuals plots reliable to evaluate the fit of the models (POPAN in particular)?

Especially in chapter 12 I do not understand why using individual capture histories the residuals are not useful ( I tried with both kinds of input files and the plot was the same, the only difference was that the capture history order on x axis was reversed).

I also found these tips on Internet but I do not know if they are correct or not. Can you take a look and tell me if this procedure is correct or if it applies only to individual covariate analysis? They also say that the residuals plots may not be reliable.

“b) Examine plot of model residuals. Gary White has told me that he does not think that the residuals are useful for examining goodness of fit for models with individual covariates. This function also does not work properly in MARK because the expected values are calculated incorrectly. I originally thought that the expected values needed to be divided by the number of observations per capture history. If that is the case, then the following directions will generate a residual plot.
(1) To examine residuals, you need to export your residuals to Excel (Output… Specific Model Output… Residuals… List Observed and Expected in Excel).
(2) Add a new column, and create corrected expected value by dividing the expected value by the number of records with the same capture history.
(a) Sort the data by capture history.
(b) Create a list of possible capture histories and the number of time each occurred; make sure this is also sorted.
(c) Use the LOOKUP function in Excel to divide the reported expected value by the number of observations.
(3) Calculate the Pearson residual
(4) Plot the residuals on a scatterplot.
c) Unfortunately, the expected values in MARK are wrong in some other way than just the number of observations per capture history. This is because there are odd patterns visible for residuals of simulated data. For fun, order your input file by capture history, and then fit the model and plot the residuals “.

I am sorry to bother you with such a long post, but I am a beginner and I do not know where else to find experts to ask for help. :(
Thank you in advance,
Nina
nlsanto2
 
Posts: 18
Joined: Wed Mar 14, 2012 5:01 am

Re: Mark residual plots

Postby abreton » Thu Jul 25, 2013 11:52 am

How are the expected values computed in MARK?


The mark help file provides the formulae, see also page 5-34 in the MARK gentle intro.

Are the residuals plots reliable to evaluate the fit of the models (POPAN in particular)?


For live-recapture models, the gentle intro suggests that in addition to Release and U-CARE,

Residual plots can also be quite helpful
(last line, page 5-40)

Note that the authors (Cooch and White) of the Gentle Intro tend to favor the Deviance Residuals in their examples, rather than Pearson Residuals. As well, the interrogation of residuals in the gentle intro is always (based on my quick scan) accompanied by an estimate of c-hat (overdispersion parameter). In other words, the residuals are considered but are not used exclusively in goodness-of-fit (GOF) examples provided in the text. This makes intuitive sense given the difficulty/challenges of devising GOF tests for capture-recapture models. The authors (Cooch and White) use what is available ... including deviance residuals in some cases. The collective evidence is then used to assess GOF.

On page 11-46 (sidebar) of the gentle intro,

One approach to assessing the fit of a model to a particular set of data is to consider the deviance residual plots.While this can prove useful - in particular, to assess lack of fit because the structure of the model is not appropriate given the data (e.g.,TSMmodels - seeChapter 7), if you try this approach for models with individual covariates, you’ll quickly run into a problem.


Once again, the authors report that deviance residuals "can be useful" ... meaning a valuable tool for assessing GOF when used with other approaches (especially an estimate of c-hat). Regarding individual covariates, the sidebar goes on to explain why you will "quickly get into trouble." And concludes,

the deviance residual plots for models with individual covariates are not generally interpretable


So, the res plot feature does not work properly in mark ... WHEN individual covariates (IC) are incorporated into the encounter histories and this is why (I suspect),

Gary White has told me that he does not think that the residuals are useful for examining goodness of fit for models with individual covariates


More subtly, it appears that the function does not work even when no ICs are integrated into the encounter histories but those histories are summarized for EACH animal rather than each unique capture history. I believe the sidebar text on page 11-47 is the reason why Cooch and White wrote,

Notice that a separate capture history is used for each tagged fish - unfortunately, this implies
that the residual plots and deviance plots in MARK cannot be used to assess goodness of fit.


... on page 12-14. However, I should note here that the classic 'dipper' dataset available from the MARK example files has one encounter history per dipper ... and residual plots appear to be valid. My suspicion (KEEP IN MIND I'm guessing here) is that when no ICs are identified when importing into MARK ... then when the user requests a 'residual plot' option then the encounter histories are grouped by each unique capture history and then integrated into the residual plot. However, this trick isn't possible when ICs are included in the encounter histories ... at least no one has been willing to strip the ICs from the histories, group the histories that remain and then make them available for a residual plot. And of course, if you chose a model that included one of those ICs it wouldn't make any sense to strip the IC ... so perhaps it's a good thing that when ICs are included the res plots are labeled "uninterpretable."

In conclusion, I don't have ALL of the answers you're searching for so hopefully someone else will weigh in. However, it's clear that if you're including ICs in your encounter histories then res plots WILL NOT be interpretable in MARK ... for any data type including POPAN. I would say ... not a big deal though ... you still have c-hat to assess GOF of your global model ... as well as information criteria for model selection.

andre
abreton
 
Posts: 111
Joined: Tue Apr 25, 2006 8:18 pm
Location: Insight Database Design and Consulting

Re: Mark residual plots

Postby nlsanto2 » Fri Jul 26, 2013 10:39 am

Andre,
thank you very much for your answer it is very helpful! :D
Nina
nlsanto2
 
Posts: 18
Joined: Wed Mar 14, 2012 5:01 am

Re: Mark residual plots

Postby abreton » Fri Jul 26, 2013 10:53 am

You're welcome Nina, best of luck with your analysis.
abreton
 
Posts: 111
Joined: Tue Apr 25, 2006 8:18 pm
Location: Insight Database Design and Consulting


Return to analysis help

Who is online

Users browsing this forum: No registered users and 3 guests

cron