www.phidot.org

Hello,

I have a data set I would like to fit with a multi-state model in RMark. I have 103 individuals and encounter histories with 189 entries. I ran a model with constant parameters and got the following error message:

Error in diag(nrow(complete.design.matrix)) :
cannot allocate vector of length 1262807296

Does this suggest that my data set is too large for RMark/MARK? Some colleagues analyzed this data set back in 2011 and had to use SURVIVE because MARK could not accommodate such long encounter histories but I was hoping that the newest version of MARK would allow for larger data sets. Are there any other options available for fitting multi-state models to a data set of this size?

Thank you very much for your help,
Javan

You didn't say how many strata you have but I'm guessing it is large. If you have k strata then you have k*(k-1)×(nocc-1) entries in DM just for Psi. Also if you have g groups then that is multiplied by g. What you are specifying is too much for the available memory. Are you using R64 and how much memory do you have?
The error is occurring before it tries to run MARK so you still don't know whether it will accept the longer capture histories. You may want to try using MARK interface with your data to see if it accepts the longer capture histories.

Hi Jeff,
Thanks for responding. I have two strata but it sounds like even that would make for a very large DM. I tried running the model 64-bit R (had previously been using 32-bit) but got the following error. I will give it a try directly with MARK though.

Error : cannot allocate vector of size 9.4 Gb
In addition: Warning messages:
1: In diag(nrow(complete.design.matrix)) :
Reached total allocation of 8106Mb: see help(memory.size)
2: In diag(nrow(complete.design.matrix)) :
Reached total allocation of 8106Mb: see help(memory.size)
3: In diag(nrow(complete.design.matrix)) :
Reached total allocation of 8106Mb: see help(memory.size)
4: In diag(nrow(complete.design.matrix)) :
Reached total allocation of 8106Mb: see help(memory.size)
Error in mark(mstrata.processed, mstrata.ddl) :
Misspecification of model or internal error in code

Thanks,
Javan

I'm confident that MARK can handle an encounter history of 189 occasions. However, before you spend a lot of time developing a design matrix, create the simplest model you can using the PIMs and then run it to be sure that your encounter histories file is valid.

If you only have 2 strata try the logit link for Psi. However, something seems wrong here. A problem this size should not need that much memory.

Hi Gary and Jeff,

Thanks for your responses. That's encouraging to hear that MARK should be able to handle my data. I will review my data and design matrix formats to see if I can find out what is causing the error.

Thanks,
Javan

Hi all,
I am new to Mark and RMark and I'm trying to run a multistrata model. I replied to this post because this is exactly the issue that I'm having and I'm curious if this was resolved and if so, how. In my data, I have 642 individuals, 91 occasions, and 11 strata. When I try to run the model, I get 2 errors:

Error : cannot allocate vector of size 1511.9 Gb
Error in mark(eh_2013_2020_2.processed, eh_2013_2020_2.ddl, model = "Multistrata") :
Misspecification of model or internal error in code

I am running 64-bit R and have 32Gb of RAM.

Any insights or suggestions are greatly appreciated.
Thanks,
Ben

Hi Ben,
You have 32 Gb of RAM, and MARK is trying to allocate of vector of size 1511.9 Gb (which requires memory to do so)....

You need to reduce the dimension of your problem. 11 strata and 91 occasions is going to involve a lot of parameters to estimate, especially with 642 individuals.

Cheers
Darryl

I figured this would be a long shot but was hoping there might be a way around this issue without reducing my data set. Sounds like that may not be possible.
Thanks

Try setting pim.type="time" for Psi in make.design.data. That will reduce the size of the design data for Psi. Also if many of your transitions between the 11 states are not feasible (set to 0) then use fixed=0 in the Psi design data for those transitions that are 0 and use mlogit0=TRUE in the call to mark or mark.wrapper. But Darryl is correct that you are going to have to limit the complexity of the model with n=642 and n=91 occasions.

--jeff