Error in memory allocation

[scha]

Hi all,

I very recently post a thread for a memory problem. viewtopic.php?f=21&t=2777

To briefly recall I try to run simple 'know-fate model' on released birds. I use 4 states and capture histories with 72 recaptures.

A model which assume a constant value for all paramaters so 4 parameters does not run due to memory problem

* * WARNING * * Error number 5014 from memory allocation of VCRMAT.

* * WARNING * * Attempt to allocate an allocated object

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x6DBD9D in _gfortrani_backtrace
#1 0x6C61C0 in _gfortrani_backtrace_handler
#2 0x7F32214D622F
#3 0x7F3221522A76
#4 0x5C421B in estmat_
#5 0x62B21C in MAIN__ at mark.f90:0
Segmentation fault

This problem is already known (see viewtopic.php?f=21&t=2741&p=8717&hilit=memory+allocation+of+VCRMAT#p8717) . But the solutions that are given is this post do not work for me. For exemple, PSI is the parameter of interest for me so I cannot fix this parameter and I cannot use use pim.type="constant" for all model.

More strange, when I try to run this simple model on the bigest computer of my lab (24G of memory),the error remains the same (both computers I have tried are under linux machine amd64)

I am pretty sure that this problem may be due to the triangular matrices of Psi which assume each real parameters for each triangular matrix to be unique (representing more than 100000). When I assume PSI to be constant, the PIM contain the same huge number of different parameters. Nevertheless, only four parameter are estimate...

This result to big .out file (<20Mo) just for parameter

It seems that no real solution exist! Someone could confirm ?

Best

Stephane

[jlaake]

You describe your analysis as known-fate but then talk about Psi which is not a parameter in the known fate model. Is this a multi-state model where you set p=1. You are correct that Psi is the problem. It is using an mlogit link and all of the parameters must differ - I can't simplify PIM structure with the way the code is written in MARK. The model is running but when MARK gets to the var-cov matrix of the real parameters it requires too much memory and fails. If you only have time variation in Psi then you can set pim-type="time". Unfortunately I've not had time to implement pim.type="age". Beyond that you can try setting link="logit" for Psi and it can then simplify the parameter structure but sometimes that can lead to numerical issues because it depends on the penalty in the MARK code to keep sum of transitions=1. If you go with logit, then I suggest you use simulated annealing.

--jeff

[scha]

Hi ,

Thanks for the answer. You're right, is not really a known fate model but a multi-state model where I set p=1.

The states I used correspond to habitat suitabilities for birds and these transition probabilities will be the core of our study. So, we want to 'play' with PSI to understand: what can biologically influence the transitions between habitats. Time variation in Psi is interesting but we also need to more precisely model Psi (according age and several groups). So unfortunately, pim-type="time" will not be sufficient in our case.

I have tried with a logit link ans it runs for the model where all paramaters are constant. Nevertheless, when I try
Psi.str=list(formula=~-1+stratum:tostratum,link="logit")
I have the "error Numerical convergence never reached.". You anticipated this potential problem and you suggested me to use 'simulated annealing'. I am really sorry but I don't understand what does it mean.

Is it similar to the repetition of the same model with changing the initial parameters?


Stephane

[jlaake]

Add options="SIMANNEAL" in the mark or mark.wrapper function and it will use simulated annealing.

If that doesn't work you can try the marked package on CRAN. I've not tried the MS model with this size of problem and it may be slow but it may be your only alternative if you can't get RMark/MARK to work for you. You'll find the functions and arguments to be very similar to RMark. You'll use the crm function in place of the mark function. The model you want is either "MSCJS" or "HMMCJS". The former can be used with ADMB which is faster. If you don't have ADMB to build the generic executable, you can install version 1.1.6 of marked from https://drive.google.com/folderview?id=0B77g1ScdUwVeRkMxbHZZT3I3UlU&usp=sharing which includes the executable.

[bootzies]

Hello.
I am having the same problem as Stephanie when generating the VCV matrix for Psi in my multistate models.
At the moment the models I am attempting are relatively simple (for example: S~time, p~1, Psi~1), however I have almost 6000 individuals and 30 (annual) capture locations, with 6 states; so if unique estimates are being made for all Psi, then that will obviously cause a problem!
After failing in Windows, I tried running the models in Linux with the same results (I get the same error as posted at the start of this thread). I have been monitoring the CPU and memory useage and I'm pretty sure that my hardware is not the issue (it has much greater capability than what is being used by MARK and R).
I was wondering whether you managed to get your models running in the end Stephanie? And if you did - how did you acheive this?!
Looking forward to hearing some good news

[scha]

Hi bootzies,

You're righ with 6 states and 30 capture occasions you will have a large number of parameter for PSI. As Jeff wrote in this thread, if you use the link to mlogit PSI, you will force all parameters PSI be different.

Thanks to Jeff, I managed to run my models.

I used a logit link for PSI and Add options="SIMANNEAL" as option in mark.wrapper (as Jeff told me to do). It worked pretty well.

Stephane