different results from different DMs for same model?

[prldb]

I have a CJS model for which survival depends on a group with three levels and one time-related covariate and recapture p has a sex*time dependence. I am looking at reduced models for survival so in the design matrix (DM) I have used an identity matrix for the recapture dependence since it will be fixed and for the survival part of the DM I used the format illustrated on p. 6-54 of the MARK book (11'th edition). MARK returns the correct parameter count and all looks reasonable for the rest of the output. All still looks good when I remove the interaction terms to obtain an additive model for survival (i.e., remove both the interaction columns in the illustration on p. 6-54). Playing around with MARK, I altered the DM for survival by setting the 1's in the intercept column to 0's for those rows in which there is a 1 in one of the columns coding for the two non-reference levels of the three-level group. Doesn't this simply alter the values of B1, B2, and B3 giving a different parametrizatrion of the same model? Yet MARK returns a very different deviance (-2log-likelihood) for this modified DM and the real parameter estimates themselves are different. My impression from reading Ch. 6 of the MARK book is that different DM's for the same model should give the same deviance and same real parameter estimates (or at least nearly so), though they may give different parameter counts. In fact, the modified DM resulted in one less parameter count as MARK could not estimate the recapture parameter for males for the final occasion (it reported a value of 1 with 0 SE wheras the original DM returned a sensible estimate for this parameter and its SE). Different parameter estimates will alter the likelihood but it's not clear to me why I'm getting different real parameter estimates (or for that matter, why the modified DM has trouble with that one recapture parameter when the original DM didn't). When I performed this same exercise after reducing the dependence of recapture p (in the DM I simply identified those recapture estimates that were very similar in magnitude so as increase precision of the estimates; in particular, p's for the final occasion were identified with other p's thereby removing any possible confounding), the two different DM's for the survival structure with the same (reduced) recapture structure did indeed return exactly the same results (deviances, parameter count, real parameter estimates) as I would have expected. So why does MARK give different results when the structure for p is more complicated? (Thinking about this problem has led me to be puzzled about how MARK is estimating p for the final occasion. The time-related covariate presumably gives an estimate of survival for the final occasion, thereby removing the confounding of survival and recapture for the final occasion, except that in this model there are three survival parameters for the final occasion, none of which correspond to the sex difference in recapture, so what does MARK do?) (I redid the exercise three times to make sure I wasn't making a dumb mistake in modifying the DM.)

[cooch]

I have a CJS model for which survival depends on a group with three levels and one time-related covariate and recapture p has a sex*time dependence. I am looking at reduced models for survival so in the design matrix (DM) I have used an identity matrix for the recapture dependence since it will be fixed and for the survival part of the DM I used the format illustrated on p. 6-54 of the MARK book (11'th edition).

Time to update to 12th edition - the material you reference is now on p. 52.

MARK returns the correct parameter count and all looks reasonable for the rest of the output. All still looks good when I remove the interaction terms to obtain an additive model for survival (i.e., remove both the interaction columns in the illustration on p. 6-54). Playing around with MARK, I altered the DM for survival by setting the 1's in the intercept column to 0's for those rows in which there is a 1 in one of the columns coding for the two non-reference levels of the three-level group. Doesn't this simply alter the values of B1, B2, and B3 giving a different parametrizatrion of the same model?

<snip...>

Not quite sure what you're trying to do. You shouldn't generally be changing the value of the coding for the intercept - you can (on occasionally want to) change the coding for the levels of the group (attribute) coding. For example, take the following example data - 3 groups, 5 sampling occasions:

Code: Select all

10100 39 44 30;
10001 16 2 18;
11000 67 35 50;
10010 18 4 16;
10000 247 355 298;
11110 11 6 2;
11101 10 1 7;
10101 13 1 9;
11001 16 0 8;
11011 4 0 5;
11100 23 25 22;
11111 3 0 3;
10110 20 12 12;
11010 8 5 13;
10011 4 5 4;
10111 1 5 3;
01000 204 290 248;
01100 111 123 97;
01010 34 21 42;
01101 37 9 29;
01001 26 5 35;
01110 47 37 21;
01111 18 11 11;
01011 23 4 17;
00110 105 95 93;
00100 284 360 293;
00101 79 27 80;
00111 32 18 34;
00010 380 435 362;
00011 120 65 138;


Here is one design matrix coding for a (group x linear) covariate interaction for apparent survival. Here, we use the default (in MARK) reference coding, where the final level of the treatment is the 'control'. Column 1 is the intercept, column 2-3 code for the groups (k=3 levels, k-1 columns = k-1 = 2 df), the linear covariate (with values 1,3,4,2) in column 4, and then the final two columns showing the interactions.

Code: Select all

1  1  0  1  1  0
1  1  0  3  3  0
1  1  0  4  4  0
1  1  0  2  2  0
1  0  1  1  0  1
1  0  1  3  0  3
1  0  1  4  0  4
1  0  1  2  0  2
1  0  0  1  0  0
1  0  0  3  0  0
1  0  0  4  0  0
1  0  0  2  0  0


Now, here is a DM which simply changes the reference cdoing to (in this example) the second level).

Code: Select all

1  1  0  1  1  0
1  1  0  3  3  0
1  1  0  4  4  0
1  1  0  2  2  0
1  0  0  1  0  0
1  0  0  3  0  0
1  0  0  4  0  0
1  0  0  2  0  0
1  0  1  1  0  1
1  0  1  3  0  3
1  0  1  4  0  4
1  0  1  2  0  2


They will give you *exactly* the same model deviance (as they should) - the reconstituted estimates (on the real probability scale) will be identical. The only thing which differs between the two DM's are the beta's, which reflect differences of a given level of the attribute group relative to the control. Change the control level, and the beta's change.

Now, if you fit different DM's which you know are equivalent structurally (like the two above), and you get different model deviances, then the problem (in so many words) lies in your data. Nothing more than that.

[cooch]

You might also want to read the - sidebar - material starting on p. 75 of Chapter 6 (12th edition). There is occasionally an issue if you make the reference coding level one that involves a parameter which might be confounded, due to the structure of the model. For example, using the final time interval in a CJS model as the reference level of 'time' is generally not good idea, even though that is the default structure in MARK.

[prldb]

My tampering with the DM is not biologically motivated but just experimenting with what I think are different parametrizations of the same model to test my understanding of DMs. Mathematically, the DM on p. 6-52 (12th Ed.) (for the additive model without any interactions) amounts to phi's for the reference level of the group being given by the form B1+covariate (on the logit scale), phi's for the second level of the group given by the form B1+B2+covariate, and phi's for the third level of the group given by the form B1+B3 + covariate. Mathematically, can't one absorb B1 into each of B2 and B3 (in effect redefining what B2 and B3 mean). The correspondng DM would be like that on p. 6-52 but the 1's in the B1 column for rows refering to phi's in levels two and three of the group would be set to zero. Isn't this just a slightly more complicated analogue (in the presence of the covariate) of the different DMs described on p. 6-8? There one has a 4-level group (and no time) and one can use a DM with a common intercept across levels or an identity DM. Isn't what I'm describing in effect an identity DM for the group part of the dependence in survival? So shouldn't I get the same deviance rather than a deviance some 50 units higher? Am I overlooking some way in which this simple change to the DM actually alters the model rather than just the parametrization? Or is there something about this parametrization that leads MARK to return a false result?

I am aware from the book by Amstrup et al. of the other issue you mention and do typically use the parametrization they recommend to avoid having parameters refering to the final occasion as reference level. In the above instance, I am using the identity DM for the p(sex*time) part of the model so that shouldn't be an issue here.

[cooch]

My tampering with the DM is not biologically motivated but just experimenting with what I think are different parametrizations of the same model to test my understanding of DMs.

If they are equivalent, then model deviance will be the same. If the deviance's are not the same, then they're not equivalent. Period. The question, then, is what *your* DM's are actually coding for.

Mathematically, the DM on p. 6-52 (12th Ed.) (for the additive model without any interactions) amounts to phi's for the reference level of the group being given by the form B1+covariate (on the logit scale), phi's for the second level of the group given by the form B1+B2+covariate, and phi's for the third level of the group given by the form B1+B3 + covariate. Mathematically, can't one absorb B1 into each of B2 and B3 (in effect redefining what B2 and B3 mean). The correspondng DM would be like that on p. 6-52 but the 1's in the B1 column for rows refering to phi's in levels two and three of the group would be set to zero. Isn't this just a slightly more complicated analogue (in the presence of the covariate) of the different DMs described on p. 6-8? There one has a 4-level group (and no time) and one can use a DM with a common intercept across levels or an identity DM. Isn't what I'm describing in effect an identity DM for the group part of the dependence in survival? So shouldn't I get the same deviance rather than a deviance some 50 units higher? Am I overlooking some way in which this simple change to the DM actually alters the model rather than just the parametrization? Or is there something about this parametrization that leads MARK to return a false result?

Can't help you unless you show us what the DM is you're attempting to describe in words. Take the example data set I posted, and show us - exactly, by writing them out, or pointing to a screen capture - what the DM's look like that you're thinking about.

I am aware from the book by Amstrup et al. of the other issue you mention....

FYI, been in the MARK book since 2002 edition. Amstrup et al. published 2006.

[prldb]

Okay, here is the DM for survival from p. 6-52 of the MARK book with the interaction columns removed to give an additive model, where B4 is the column of covariate values. My DM1 is analogous to this DM (for recapture p the structure is sex*time and I use the identity matrix for that part of the DM as it remains fixed; the 3-level group in the structure for survival is not related to sex)

B1 B2 B3 B4
1 0 0 1.1
1 0 0 0.2
1 0 0 3.4
1 0 0 4.1
1 0 1 1.1
1 0 1 0.2
1 0 1 3.4
1 0 1 4.1
1 1 0 1.1
1 1 0 0.2
1 1 0 3.4
1 1 0 4.1

My modified design matrix for survival is DM2:

B1 B2 B3 B4
1 0 0 1.1
1 0 0 0.2
1 0 0 3.4
1 0 0 4.1
0 0 1 1.1
0 0 1 0.2
0 0 1 3.4
0 0 1 4.1
0 1 0 1.1
0 1 0 0.2
0 1 0 3.4
0 1 0 4.1

So all I did was set to 0 the 1's in column B1 for those rows in which either B2 or B3 was 1. I would think DM1 and DM2 are just two different paramerizations of the same model as all one has done is redefine what B2 and B3 will mean (as in the examples of alternative parameritzations on p. 6-8). Yet I get very different deviances.

On the oher hand, I have discovered that if instead of using the identity DM for the recaptue part of the DM, I use the more usual DM for a sex*time structure, as on p. 6-9 of the MARK book except modified so that the last time occasion is not the reference coding level, e.g.,

1 1 0 0 0 0 0 0
1 1 1 0 0 1 0 0
1 1 0 1 0 0 1 0
1 1 0 0 1 0 0 1
1 0 0 0 0 0 0 0
1 0 1 0 0 0 0 0
1 0 0 1 0 0 0 0
1 0 0 0 1 0 0 0

with both DM1 and DM2 for the survival parameters as described above then I do indeed get the same deviances (and both agree with DM2 in combination with the identity design for recapture; the odd one out is DM1 with the identity design for recapture, which in addition to giving a different deviance assigned a value of 1 with 0 SE to the recapture parameter for males at the final occasion so, for some reason, estimating this parameter is a problem in this design but not the others and perhaps this results in the different deviances)

[cooch]

Okay, here is the DM for survival from p. 6-52 of the MARK book with the interaction columns removed to give an additive model, where B4 is the column of covariate values. My DM1 is analogous to this DM (for recapture p the structure is sex*time and I use the identity matrix for that part of the DM as it remains fixed; the 3-level group in the structure for survival is not related to sex)

B1 B2 B3 B4
1 0 0 1.1
1 0 0 0.2
1 0 0 3.4
1 0 0 4.1
1 0 1 1.1
1 0 1 0.2
1 0 1 3.4
1 0 1 4.1
1 1 0 1.1
1 1 0 0.2
1 1 0 3.4
1 1 0 4.1

My modified design matrix for survival is DM2:

B1 B2 B3 B4
1 0 0 1.1
1 0 0 0.2
1 0 0 3.4
1 0 0 4.1
0 0 1 1.1
0 0 1 0.2
0 0 1 3.4
0 0 1 4.1
0 1 0 1.1
0 1 0 0.2
0 1 0 3.4
0 1 0 4.1

So all I did was set to 0 the 1's in column B1 for those rows in which either B2 or B3 was 1. I would think DM1 and DM2 are just two different paramerizations of the same model as all one has done is redefine what B2 and B3 will mean (as in the examples of alternative parameritzations on p. 6-8). Yet I get very different deviances.

Running it against sample data set I provided (please, in future, always use a common reference data set - if you want someone to help, they need an empirical frame of reference), either paramterization gives identical deviances, regardless of how the encounter parameter p is modeled.

Using the following additive structure,

Code: Select all

1  0  0  1
1  0  0  3
1  0  0  4
1  0  0  2 
1  0  1  1
1  0  1  3
1  0  1  4 
1  0  1  2
1  1  0  1
1  1  0  3
1  1  0  4 
1  1  0  2 


model deviance is 77.0787 - 16 parameters. Parameter estimates for apparent survival are:

Code: Select all

     1:Phi                   0.6851846       0.0258639       0.6324384       0.7335505
     2:Phi                   0.8123874       0.0172813       0.7761493       0.8439386
     3:Phi                   0.8593070       0.0184339       0.8191645       0.8917165
     4:Phi                   0.7542946       0.0184393       0.7163942       0.7886246
     5:Phi                   0.4304806       0.0270593       0.3784299       0.4841144
     6:Phi                   0.6006095       0.0243111       0.5521505       0.6471756
     7:Phi                   0.6796042       0.0311524       0.6157472       0.7373753
     8:Phi                   0.5160089       0.0216414       0.4735784       0.5582098
     9:Phi                   0.6208545       0.0285790       0.5634596       0.6750551
    10:Phi                   0.7651406       0.0191922       0.7254667       0.8006565
    11:Phi                   0.8212766       0.0212161       0.7758629       0.8591591
    12:Phi                   0.6978593       0.0203946       0.6564568       0.7362767               


Now using your coding approach,

Code: Select all

1  0  0  1
1  0  0  3
1  0  0  4
1  0  0  2 
0  0  1  1
0  0  1  3
0  0  1  4 
0  0  1  2
0  1  0  1
0  1  0  3
0  1  0  4 
0  1  0  2 


you get the exact same model deviance (77.0787), and the same reconstituted estimates for apparent survival:

Code: Select all

     1:Phi                   0.6851843       0.0258639       0.6324381       0.7335503
     2:Phi                   0.8123876       0.0172813       0.7761495       0.8439388
     3:Phi                   0.8593073       0.0184339       0.8191648       0.8917168
     4:Phi                   0.7542947       0.0184393       0.7163942       0.7886246
     5:Phi                   0.4304803       0.0270593       0.3784296       0.4841142
     6:Phi                   0.6006099       0.0243111       0.5521508       0.6471761
     7:Phi                   0.6796048       0.0311525       0.6157477       0.7373760
     8:Phi                   0.5160089       0.0216414       0.4735784       0.5582098
     9:Phi                   0.6208540       0.0285791       0.5634589       0.6750547
    10:Phi                   0.7651407       0.0191922       0.7254668       0.8006566
    11:Phi                   0.8212769       0.0212161       0.7758632       0.8591594
    12:Phi                   0.6978591       0.0203946       0.6564565       0.7362766             


So, if you're getting different deviances, then you're either doing something wrong at some point, or its the data set you're 'testing' against.

[cooch]

If you want to try out the data I posted, click here to download a .zip archive containing the data, and asociated MARK files (.fpt and .dbf).

[prldb]

Mnay thanks. That confirms my understanding of how things should behave.

[cooch]

Many thanks. That confirms my understanding of how things should behave.

Good enough...