BACI contrasts - calculating lsmeans from Mark results

Forum for discussion of general questions related to study design and/or analysis of existing data - software neutral.

BACI contrasts - calculating lsmeans from Mark results

Postby abmiller » Tue Nov 17, 2015 3:39 pm

I am working on analysis for a before-after-control-impact study and have been using the occupancy model in Rmark to calculate occupancy of ten species in an area. As this is a BACI study, I would like to calculate the BACI contrast to answer the question, "Is the amount of change between before and after phases significantly different for the impact site as compared with the control site?".

I have been able to do this using count data run with a glm poisson regression model, as well as with the zero-inflated poisson regression model. For the zero-inflated poisson regression model, the developer of the lsmeans package helped me by supplying a script to calculate lsmeans from zero-inflated poisson regression results, which was not already a capability of the lsmeans package.

I imagine others have used occupancy results to answer BACI questions in the past. If anyone has other suggestions for a method I could use to answer this type of question I would be happy to hear that as well. I've tried splitting my data into groups based on site and determining if there is a significant difference between phases (before/after) (see: http://www.phidot.org/forum/viewtopic.php?f=1&t=1906&p=5670&hilit=baci#p5670), but this still leaves the question unanswered of whether there was a difference in the difference between sites. The data were collected over one year, so seasonality is an issue in the analysis. Camera traps were used to collect data.

The model I'm using in Rmark is:
Code: Select all
species.models=mark(species, model="Occupancy", group = c("Zone", "Phase"), model.parameters = list(p=list(formula=~WeekDay+DetectionDist), Psi=list(formula=~Zone+Phase)), invisible=FALSE)
where Zone = Control or Impact (factor); Phase = Before or After (factor); WeekDay = numerical value for the day of the entire project on which the survey occurred (values ranging from 0-370); DetectionDist = maximum distance at which the camera was triggered on setup.


And the code I'm using in R to calculate the contrasts from zero-inflated poisson regression is below. Perhaps this helps explain my question further. {ZP = Zone+Phase factor;

Code: Select all
recover.data.zeroinfl = lsmeans:::recover.data.lm

lsm.basis.zeroinfl = function(object, trms, xlev, grid, ...) {
    m = model.frame(trms, grid, na.action = na.pass, xlev = xlev)
    contr = object$contrasts
    contr = contr[setdiff(names(contr), c("zero", "count"))]
    X = model.matrix(trms, m, contrasts.arg = contr)
    use = seq_len(ncol(X)) # use only the elts related to response
    bhat = coef(object)[use]
    V = vcov(object, ...)[use, use]
    nbasis = estimability::all.estble
    misc = list(tran = "log", inv.lbl = "rate")
    dffun = function(k, dfargs) NA
    dfargs = list()
    list(X = X, bhat = bhat, nbasis = nbasis, V = V,
        dffun = dffun, dfargs = dfargs, misc = misc)
}

bact <- read.csv("C:/....CountCameraDay_AllSp.csv",header=TRUE)

summary(m1 <- zeroinfl(Skunk ~ ZP + ProjectDay + DetectionDist + offset(SurveyLength))|ZP + ProjectDay + DetectionDist, data = bact))

result.lsmo.SP1 <- lsmeans::lsmeans(m1, ~ZP+ProjectDay+DetectionDist)
summary(result.lsmo.SP1)

contrast(result.lsmo.SP1, list(bact=c(1, -1, -1, 1)))
confint(contrast(result.lsmo.SP1, list(bact=c(1, -1, -1, 1))))



Thanks in advance for any suggestions you can offer!
abmiller
 
Posts: 3
Joined: Mon Nov 16, 2015 5:43 pm

Re: BACI contrasts - calculating lsmeans from Mark results

Postby jlaake » Tue Nov 17, 2015 4:39 pm

As this is a BACI study, I would like to calculate the BACI contrast to answer the question, "Is the amount of change between before and after phases significantly different for the impact site as compared with the control site?".


Based on the above goal, you fitted the wrong model. You fitted an additive model (~Zone+Phase) for occupancy which assumes that the difference of before and after is the same for Control/Impact. What you should have used was ~Zone*Phase which allows the Phase value(Before/After) to differ between Control/Impact. The value you want will then be the interaction term between zone and phase and in the beta coefficients/summary you will get an estimate and a standard error and from that you can construct a test or just see if confidence interval includes 0 if you want alpha=0.05.

Alternatively a comparison using AIC of Zone+Phase and Zone*Phase models is equivalent to a chi-square test with alpha=0.15 because they will only differ by one parameter.

--jeff
jlaake
 
Posts: 1479
Joined: Fri May 12, 2006 12:50 pm
Location: Escondido, CA

Re: BACI contrasts - calculating lsmeans from Mark results

Postby abmiller » Wed Nov 18, 2015 11:49 am

Thanks so much for this explanation. I've changed the occupancy model to include Zone*Phase instead of Zone+Phase. I'd like to confirm if I'm interpreting this correctly and follow up with another question.

Here is the output I get for the second most common species in the data, Squirrel:

Code: Select all
> BEAR.models.bact=mark(bact, model="Occupancy", group=c("Zone", "Phase"), model.parameters=list
+   (p=list(formula=~WeekDay+DetectionDist+Trail),Psi=list(formula=~Zone*Phase)), invisible=FALSE)
STOP NORMAL EXIT

Output summary for Occupancy model
Name : p(~WeekDay + DetectionDist + Trail)Psi(~Zone * Phase)

Npar :  8
-2lnL:  2015.356
AICc :  2032.392

Beta
                      estimate           se        lcl        ucl
p:(Intercept)    -1.7343132000 0.2169102000 -2.1594572 -1.3091692
p:WeekDay        -0.0001062123 0.0005992767 -0.0012808  0.0010684
p:DetectionDist  -0.0098418000 0.0115468000 -0.0324735  0.0127900
p:Trail           0.0124737000 0.1788772000 -0.3381257  0.3630731
Psi:(Intercept)   0.3855771000 0.3249939000 -0.2514110  1.0225652
Psi:ZoneT         0.8919546000 0.5551349000 -0.1961098  1.9800190
Psi:PhaseO        0.8617738000 0.5747011000 -0.2646404  1.9881879
Psi:ZoneT:PhaseO -0.3259003000 0.9570754000 -2.2017682  1.5499676


Real Parameter p
                          1         2         3         4         5         6
Group:ZoneC.PhaseB 0.130517 0.1305059 0.1304948 0.1304837 0.1304727 0.1304616
Group:ZoneT.PhaseB 0.130517 0.1305059 0.1304948 0.1304837 0.1304727 0.1304616
Group:ZoneC.PhaseO 0.130517 0.1305059 0.1304948 0.1304837 0.1304727 0.1304616
Group:ZoneT.PhaseO 0.130517 0.1305059 0.1304948 0.1304837 0.1304727 0.1304616
                           7         8         9        10        11        12
Group:ZoneC.PhaseB 0.1304505 0.1304394 0.1304284 0.1304173 0.1304062 0.1303952
Group:ZoneT.PhaseB 0.1304505 0.1304394 0.1304284 0.1304173 0.1304062 0.1303952
Group:ZoneC.PhaseO 0.1304505 0.1304394 0.1304284 0.1304173 0.1304062 0.1303952
Group:ZoneT.PhaseO 0.1304505 0.1304394 0.1304284 0.1304173 0.1304062 0.1303952
                          13       14       15        16        17        18
Group:ZoneC.PhaseB 0.1303841 0.130373 0.130362 0.1301915 0.1300662 0.1301683
Group:ZoneT.PhaseB 0.1303841 0.130373 0.130362 0.1301915 0.1300662 0.1301683
Group:ZoneC.PhaseO 0.1303841 0.130373 0.130362 0.1301915 0.1300662 0.1301683
Group:ZoneT.PhaseO 0.1303841 0.130373 0.130362 0.1301915 0.1300662 0.1301683
                          19        20       21       22        23        24
Group:ZoneC.PhaseB 0.1304077 0.1302241 0.130237 0.130261 0.1310054 0.1310997
Group:ZoneT.PhaseB 0.1304077 0.1302241 0.130237 0.130261 0.1310054 0.1310997
Group:ZoneC.PhaseO 0.1304077 0.1302241 0.130237 0.130261 0.1310054 0.1310997
Group:ZoneT.PhaseO 0.1304077 0.1302241 0.130237 0.130261 0.1310054 0.1310997
                          25        26        27        28        29        30
Group:ZoneC.PhaseB 0.1319377 0.1319356 0.1322924 0.1322915 0.1322906 0.1322897
Group:ZoneT.PhaseB 0.1319377 0.1319356 0.1322924 0.1322915 0.1322906 0.1322897
Group:ZoneC.PhaseO 0.1319377 0.1319356 0.1322924 0.1322915 0.1322906 0.1322897
Group:ZoneT.PhaseO 0.1319377 0.1319356 0.1322924 0.1322915 0.1322906 0.1322897
                          31        32       33        34        35        36
Group:ZoneC.PhaseB 0.1322888 0.1322879 0.132287 0.1322861 0.1322852 0.1322843
Group:ZoneT.PhaseB 0.1322888 0.1322879 0.132287 0.1322861 0.1322852 0.1322843
Group:ZoneC.PhaseO 0.1322888 0.1322879 0.132287 0.1322861 0.1322852 0.1322843
Group:ZoneT.PhaseO 0.1322888 0.1322879 0.132287 0.1322861 0.1322852 0.1322843
                          37        38        39        40
Group:ZoneC.PhaseB 0.1322834 0.1322825 0.1322816 0.1324118
Group:ZoneT.PhaseB 0.1322834 0.1322825 0.1322816 0.1324118
Group:ZoneC.PhaseO 0.1322834 0.1322825 0.1322816 0.1324118
Group:ZoneT.PhaseO 0.1322834 0.1322825 0.1322816 0.1324118


Real Parameter Psi
                           1
Group:ZoneC.PhaseB 0.5952175
Group:ZoneT.PhaseB 0.7820293
Group:ZoneC.PhaseO 0.7768410
Group:ZoneT.PhaseO 0.8597729
>
 


I believe this means that based on the beta estimate and 95% confidence interval for the interaction factor, Psi:ZoneT:PhaseO, there is no effect of the treatment on squirrel occupancy. Does this sound correct?


My follow up question is regarding the results for deer, the most common species in the analysis. Again the 95% CI of the interaction factor includes 0, which should mean that there is no effect of the treatment on deer occupancy. My question is regarding the occupancy estimates for deer, which include an standard error of 0.00000. The output from the occupancy model is below. I notice there is an extra line before the output is shown, "Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL". I've pasted the results of the psi estimates from the Mark output below the R output, which shows the SE of 0.00000.


Code: Select all
> BEAR.models.bact=mark(bact, model="Occupancy", group=c("Zone", "Phase"), model.parameters=list
+   (p=list(formula=~WeekDay+DetectionDist+Trail),Psi=list(formula=~Zone*Phase)), invisible=FALSE)
Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
STOP NORMAL EXIT

Note: only 6 parameters counted of 8 specified parameters

AICc and parameter count have been adjusted upward


Output summary for Occupancy model
Name : p(~WeekDay + DetectionDist + Trail)Psi(~Zone * Phase)

Npar :  8  (unadjusted=6)
-2lnL:  3917.331
AICc :  3934.367  (unadjusted=3929.9268)

Beta
                    estimate           se          lcl         ucl
p:(Intercept)     -0.9000717 1.274466e-01   -1.1498670  -0.6502764
p:WeekDay         -0.0028073 3.676578e-04   -0.0035280  -0.0020867
p:DetectionDist    0.0412056 6.603000e-03    0.0282636   0.0541476
p:Trail            0.2518846 1.077982e-01    0.0406001   0.4631691
Psi:(Intercept)   21.4507180 0.000000e+00   21.4507180  21.4507180
Psi:ZoneT        -18.5465770 0.000000e+00  -18.5465770 -18.5465770
Psi:PhaseO       -20.2062490 0.000000e+00  -20.2062490 -20.2062490
Psi:ZoneT:PhaseO  33.5957260 2.878444e+02 -530.5792700 597.7707200


Real Parameter p
                           1         2         3         4         5         6
Group:ZoneC.PhaseB 0.3426543 0.3420734 0.3414931 0.3409132 0.3403338 0.3397549
Group:ZoneT.PhaseB 0.3426543 0.3420734 0.3414931 0.3409132 0.3403338 0.3397549
Group:ZoneC.PhaseO 0.3426543 0.3420734 0.3414931 0.3409132 0.3403338 0.3397549
Group:ZoneT.PhaseO 0.3426543 0.3420734 0.3414931 0.3409132 0.3403338 0.3397549
                           7         8        9       10        11        12
Group:ZoneC.PhaseB 0.3391764 0.3385985 0.338021 0.337444 0.3368674 0.3362914
Group:ZoneT.PhaseB 0.3391764 0.3385985 0.338021 0.337444 0.3368674 0.3362914
Group:ZoneC.PhaseO 0.3391764 0.3385985 0.338021 0.337444 0.3368674 0.3362914
Group:ZoneT.PhaseO 0.3391764 0.3385985 0.338021 0.337444 0.3368674 0.3362914
                          13        14        15        16        17        18
Group:ZoneC.PhaseB 0.3357159 0.3351408 0.3345662 0.3257738 0.3193782 0.3245834
Group:ZoneT.PhaseB 0.3357159 0.3351408 0.3345662 0.3257738 0.3193782 0.3245834
Group:ZoneC.PhaseO 0.3357159 0.3351408 0.3345662 0.3257738 0.3193782 0.3245834
Group:ZoneT.PhaseO 0.3357159 0.3351408 0.3345662 0.3257738 0.3193782 0.3245834
                          19        20        21        22        23        24
Group:ZoneC.PhaseB 0.3369437 0.3274425 0.3281071 0.3293461 0.3686752 0.3737845
Group:ZoneT.PhaseB 0.3369437 0.3274425 0.3281071 0.3293461 0.3686752 0.3737845
Group:ZoneC.PhaseO 0.3369437 0.3274425 0.3281071 0.3293461 0.3686752 0.3737845
Group:ZoneT.PhaseO 0.3369437 0.3274425 0.3281071 0.3293461 0.3686752 0.3737845
                          25        26       27        28        29        30
Group:ZoneC.PhaseB 0.4201641 0.4200486 0.440206 0.4401545 0.4401031 0.4400517
Group:ZoneT.PhaseB 0.4201641 0.4200486 0.440206 0.4401545 0.4401031 0.4400517
Group:ZoneC.PhaseO 0.4201641 0.4200486 0.440206 0.4401545 0.4401031 0.4400517
Group:ZoneT.PhaseO 0.4201641 0.4200486 0.440206 0.4401545 0.4401031 0.4400517
                          31        32        33        34        35        36
Group:ZoneC.PhaseB 0.4400003 0.4399489 0.4398975 0.4398461 0.4397947 0.4397433
Group:ZoneT.PhaseB 0.4400003 0.4399489 0.4398975 0.4398461 0.4397947 0.4397433
Group:ZoneC.PhaseO 0.4400003 0.4399489 0.4398975 0.4398461 0.4397947 0.4397433
Group:ZoneT.PhaseO 0.4400003 0.4399489 0.4398975 0.4398461 0.4397947 0.4397433
                          37        38       39        40
Group:ZoneC.PhaseB 0.4396918 0.4396404 0.439589 0.4469851
Group:ZoneT.PhaseB 0.4396918 0.4396404 0.439589 0.4469851
Group:ZoneC.PhaseO 0.4396918 0.4396404 0.439589 0.4469851
Group:ZoneT.PhaseO 0.4396918 0.4396404 0.439589 0.4469851


Real Parameter Psi
                           1
Group:ZoneC.PhaseB 1.0000000
Group:ZoneT.PhaseB 0.9480508
Group:ZoneC.PhaseO 0.7763410
Group:ZoneT.PhaseO 0.9999999
>


Code: Select all
                     Estimates of Derived Parameters
 Occupancy Estimates of { p(~WeekDay + DetectionDist + Trail)Psi(~Zone * Phase) }
                                            95% Confidence Interval
 Group   Psi-hat         Standard Error      Lower           Upper
 -----   --------------  --------------  --------------  --------------
   1     1.0000000       0.0000000       1.0000000       1.0000000     
   2     0.9480508       0.0362519       0.8117528       0.9872179     
   3     0.7763410       0.0786996       0.5880981       0.8940534     
   4     0.9999999       0.6028374E-004  0.1175145E-296  1.0000000     



Thanks again for your help!
abmiller
 
Posts: 3
Joined: Mon Nov 16, 2015 5:43 pm

Re: BACI contrasts - calculating lsmeans from Mark results

Postby jlaake » Wed Nov 18, 2015 12:40 pm

I believe this means that based on the beta estimate and 95% confidence interval for the interaction factor, Psi:ZoneT:PhaseO, there is no effect of the treatment on squirrel occupancy. Does this sound correct?

Correct except for the subtle interpretation that you failed to show an effect which is not the same as there is no effect.

My follow up question is regarding the results for deer, the most common species in the analysis. Again the 95% CI of the interaction factor includes 0, which should mean that there is no effect of the treatment on deer occupancy. My question is regarding the occupancy estimates for deer, which include an standard error of 0.00000. The output from the occupancy model is below. I notice there is an extra line before the output is shown, "Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL". I've pasted the results of the psi estimates from the Mark output below the R output, which shows the SE of 0.00000.


The issue here is that occupancy is being estimated to be 1 for Zone C for phase B and O. That is what is causing the underflow error. Your std errors for occupancy beta estimates all look whacky because Zone C is the intercept. If you make Zone T the intercept level then you'll get interpretable std errors for those values but you may still get the underflow which is not a problem.

You can do that by setting ddl$Psi$Zone=relevel(ddl$Psi$Zone,"T") and re-running the model where ddl is the name of design data list. To do this you'll have to use the functions process.data and make.design.data and then feed those values to mark. See below:

bact.proc=process.data(bact, model="Occupancy", group=c("Zone", "Phase"))
bact.ddl=make.design.data(bact.proc)
bact.ddl$Psi$Zone=relevel(bact.ddl$Psi$Zone,"T")
mark(bact.proc,bact.ddl, model.parameters=list(p=list(formula=~WeekDay+DetectionDist+Trail),Psi=list(formula=~Zone*Phase)), invisible=FALSE)
jlaake
 
Posts: 1479
Joined: Fri May 12, 2006 12:50 pm
Location: Escondido, CA

Re: BACI contrasts - calculating lsmeans from Mark results

Postby abmiller » Wed Nov 18, 2015 5:40 pm

Thanks for this explanation, it's been very helpful. The code also solved the issue of SE=0 for the occupancy estimates.

However, the standard errors of the beta estimate for the interaction term is still always either 0 or very large (e.g. 5.5e+02) for deer. I've read that this occurs when psi=1 (see Welsh AH, Lindenmayer DB, Donnelly CF (2013) Fitting and Interpreting Occupancy Models. PLoS ONE 8(1): e52015.). It is normal, I believe, for psi to be 1 or near 1 for deer, since it is by far the most common species in the data. However, this is problematic for the analysis, since I am relying on the 95%CI to determine if the model shows an effect of the treatment on deer occupancy. Is there any other way I could either solve the issue of the SE of the beta estimates or use the psi estimates to answer the BACI questions?

Thanks again for your help!
abmiller
 
Posts: 3
Joined: Mon Nov 16, 2015 5:43 pm

Re: BACI contrasts - calculating lsmeans from Mark results

Postby jlaake » Wed Nov 18, 2015 5:57 pm

I'm not entirely sure what you meant in the last message, so I'm in the dark here. An alternative is to fit

~-1+Zone:Phase

It will fit four separate values for occupancy. CB,CO,TB,TO. But then you have to construct CB-CO and TB-TO which are the differences and you are then testing if CB-CO=TB-TO but that should be the same as the interaction term not=0.

From what I could tell CB=CO=1 so the difference will be 0. So you would be testing if TB-TO not = 0. This should be the same as testing the interaction term.

So we don't pester everyone on the list, send me via email the results of your previous run with releveled zone and the one above, jeff.laake@noaa.gov
jlaake
 
Posts: 1479
Joined: Fri May 12, 2006 12:50 pm
Location: Escondido, CA


Return to analysis & design questions

Who is online

Users browsing this forum: No registered users and 0 guests

cron