time to run

questions concerning analysis/theory using program MARK

time to run

Postby alij » Tue Aug 28, 2007 6:16 am

I am doing an analysis, and it has already taken over a week. I am new to mark, and I was just wondering whether this is reasonable, or whether I've done something wrong.
It's a recaptures only analysis, with 2 groups, 1 individual covariate, and nearly 4000 individuals. There are 44 capture occasions. In the design matrix there are 256 rows and 348 columns. I'm running it on a Dual Core 1.86GHz processor with 2GB of RAM. And I have not been using the computer for anything else whilst running the programme.
I'm guessing the individual covariates hugely increase the expected running time, but i was just wondering if over a week for this analysis seems within reason.
Thanks.
Ali Johnston
alij
 
Posts: 4
Joined: Fri Aug 24, 2007 11:14 am
Location: cambridge

time to run

Postby gwhite » Tue Aug 28, 2007 8:40 am

Ali:
With 348 parameters to estimate and 4800 individuals, this will take a while. You should see the iteration count incrementing, and eventually, the calculation of the VC matrix progressing.
The way to reduce the run times is to not start off with a really big model like you have, but to start with very simple models developed in the design matrix, and then using the parameter estimates from these models as starting values for more complex models. By using these starting values, you reduce the risk of getting optimization errors, plus greatly reduce the amount of time it takes for the optimization.
Gary
gwhite
 
Posts: 340
Joined: Fri May 16, 2003 9:05 am

Postby alij » Tue Aug 28, 2007 10:22 am

thanks very much.
i'll try that.
alij
 
Posts: 4
Joined: Fri Aug 24, 2007 11:14 am
Location: cambridge

Re: time to run

Postby cooch » Tue Aug 28, 2007 11:40 am

alij wrote:I am doing an analysis, and it has already taken over a week. I am new to mark, and I was just wondering whether this is reasonable, or whether I've done something wrong.
It's a recaptures only analysis, with 2 groups, 1 individual covariate, and nearly 4000 individuals. There are 44 capture occasions. In the design matrix there are 256 rows and 348 columns. I'm running it on a Dual Core 1.86GHz processor with 2GB of RAM. And I have not been using the computer for anything else whilst running the programme.
I'm guessing the individual covariates hugely increase the expected running time, but i was just wondering if over a week for this analysis seems within reason.
Thanks.
Ali Johnston


The other issue - implicit from your note - is that you're clearly trying to fit a time-dependent model. While there may be reasons to do this, most of the time such models are somewhat 'silly', since we know that there *must* be time-variation. Time variation in and of itself isn't particularly interesting (in most cases) - it's what varies over time in the environment (for example) that might influence temporal variation in a parameter that is of interest. In many cases, I've found that people fit time-dependent models 'by reflex', and haven't spent a lot of time thinking about reasonable covariates which might influence variation over time.

I mention this because there is often a fair argument that the general model isn't (or shouldn't be) necessarily a time-dependent model, but one where variation over time is constrained as a linear function of one or more temporally varying covariates that, a priori, you've thought might be reasonable. Such a constrained model will be a reduced parameter model, and as such, convergence will be achieved much more quickly.
cooch
 
Posts: 1654
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Postby alij » Mon Sep 03, 2007 11:24 am

Yes, I am fitting a time dependent model.
Initially I wanted to explore the pattern of changes over time to see if it appeared to be a trend or random variation. I am now testing nao against survival, rather than a time dependent model.
Another covariate which I would like to test is how population size is affecting apparent survival. But I'm not sure if I can put population estimates in as a covariate, as my only data for the size of the breeding pop is from the mark recapture data, and the proportion of ringed birds. If the proportion changes over time, does this make the population estimate sufficiently independent from the MR data, that it's ok to include it as a covariate??
alij
 
Posts: 4
Joined: Fri Aug 24, 2007 11:14 am
Location: cambridge


Return to analysis help

Who is online

Users browsing this forum: Bing [Bot] and 2 guests