Fortran error in RMark ORDMS estimation

[bacollier]

I thought we resolved the binary read issue on Linux machine. I don't think Bret has a problem on his Linux machine.

regards --jeff

Correct, I have not had any recent problems since we were getting that weird rawtoChar error that was a R version 2.12 (I think) issue.

bret

[bacollier]

Thanks all for comments.
Bret, I've run big ORDMS models before on this machine, but I have now expanded the number of states. Perhaps that was the straw that broke the camel's back! A little while ago I tried to run some ORDMS models on a Linux cluster, but ran into issues where Linux couldn't seem to read some of the output files, so the R output objects weren't created. Never did get that resolved, but it might be worth another try if I can get access to a cluster again.
Glenn

Glenn, my earlier problems agree roughly with what you saw, little over 1400 individuals over 10 years and I was estimating (70 to 80) unique state transitions (e.g., A to B, B to C, etc) by sex/age/time with lots of non-occurring transitions and my models were bombing when MARK got to the vc issue you hit. I have not seen the issues on the output file side in Linux you referenced. If you would like, shoot me the code you were using (offlist: bret@tamu.edu) on the Linux side when you had the problem and I will see if I can recreate the issue you were seeing.

Untested but maybe Jeff can answer help with this one; I think that if you run one model in MARK using simulated annealing, you can use the initial= function in the mark() call to set values for model parameters in subsequent model runs, maybe that would reduce the time/memory being used by getting your closer, quicker and hence take less memory?

Also, Evan reminded me yesterday that this time there is only a 64 bit version for Linux as there was no speed gain at the time Gary and he tested it for a windows machine. I work primarily on Linux and had 64 bit on there, but I have been switching computers frequently lately and running models on my windows machine as well so I got confused on what was where when (which seems to happen a lot more lately).

bret

[jlaake]

Untested but maybe Jeff can answer help with this one; I think that if you run one model in MARK using simulated annealing, you can use the initial= function in the mark() call to set values for model parameters in subsequent model runs, maybe that would reduce the time/memory being used by getting your closer, quicker and hence take less memory?

Should be faster to do the above but won't reduce memory without going to logit link.

--jeff

[jlaake]

I'll add some comments here to complete this thread after working with Gary and Glenn offlist. Glenn's original issue with the FORTRAN error was due to running out of memory in computing the variance-covariance matrix of the real parameters of which he had many. A 64 bit machine with more memory and a 64 bit version of MARK would have been one way to solve that problem.

However, in conversation with Gary we devised another way that worked for Glenn and may work for some of you that are building models with RMark. Glenn is working with multi-stratum (multi-state) models and many of his transitions are not possible and can be set to 0. In his case that would substantially reduce the number of real parameters if they could be collapsed into a single parameter and set to 0.

Now as I have posted several times before, with the mlogit link I can't simplify the PIM structure due to the way mlogits are calculated in MARK. However the one exception is when the fixed value Psi is 0 because it has no influence on the real parameter estimate. So I added 4 lines of code to RMark to set the link to logit for any mlogit parameter that is fixed to 0. With those set to logit link the code happily simplifies the PIMS for those parameters and reduces the number of real parameters before sending to MARK. Bret Collier found that it reduced his run times by an order of magnitude due to shrinking the number of rows in the design matrix.

Now that is only part of the story. When Glenn ran his models with the new version he also got the identical likelihood value and nearly identical results to within rounding error on reals. BUT, some of the intercept betas were changed substantially even though the reals were unchanged. This perplexed us for quite awhile until Glenn read in my documentation (what I had spaced out), that an exp(0) is entered for every fixed 0 value in the mlogit. When these are removed from the mlogit the betas obviously have to change to get the same real parameter value. What this means is that it all works and this may speed up execution and allow larger models for this situation.

To avoid someone stumbling on this by accident when someone re-reruns an analysis done with an older version, I'm going to add an argument mlogit0 which I'll default to FALSE to maintain the prior behavior. Only when it is set to TRUE will the mlogit 0 parameters be set to logit link to get the gain. I'll get this in the code and updated on CRAN by the end of the week.

regards --jeff