Page 1 of 1

Mark and UNIX computer cluster

PostPosted: Thu Mar 20, 2014 5:01 pm
by kris_deer
Hi there,

To save up time, I am using Mark for Linux (version ) and RMark in command batch on a computer cluster (Grid Engine) - doing so let me adjut several models as a same time and free my own machine. Note that I am not trying to use 'real' parallel computing within one job, just running several instances of Mark for the different jobs (model fit).

Everything has been working like a charm until today. The sys admin killed all my jobs, pointing out that each running job uses between 1500% and 2000% CPU (~ 15 cores) - usually 1 one job means 1 core. Actually, Mark behaves like it was able to use all available cores for each job on given cluster. I could perfectly live with this myself but the admin does not see it that way - at all.

So here I am. Has anyone experienced a similar 'problem'? Is it a feature of Mark? Any tips on how to handle this problem and force Mark to use 1 core at a time would be greatly appreciated. On my local linux box (Debian Jessie), Mark is always confined to one core and works as I expect. I apologize if you guys think this is not related to Mark but running purely R jobs do not lead to the same issue though.

Thanks

Kris

Re: Mark and UNIX computer cluster

PostPosted: Thu Mar 20, 2014 5:09 pm
by jlaake
Ouch. At least they could have given you a warning. By default MARK.exe uses parallel computing and will take #cpus-1 to do so. You can play nice by setting the threads argument for mark or mark.wrapper to some smaller value such that your sys.admin won't kill your jobs.

--jeff

Re: Mark and UNIX computer cluster

PostPosted: Thu Mar 20, 2014 5:22 pm
by kris_deer
Thank you Jeff for this swift answer. This is explicit in mark R help, indeed (RT*M). Will set this thread argument to 2 or even 3 on the cluster :evil: ... You guys are doing a fantastic job! The last Linux version rocks.

Kris