3
$\begingroup$

I am currently running canu on an LSF Linux server using the following script called assemble.sh:

canu -p species -d output genomeSize=100m -pacbio-raw subreads.fastq

I then submit this script with the following in a computer node:

bsub -q basement -E 'test -e /nfs/users/nfs_c/cr' -R "select[mem>25000] rusage[mem=25000]" -M25000 -o canu.o -e canu.e -J canu 'bash assemble.sh' -n8 -R"span[hosts=1]"

On my laptop, using 8 cores and 16 GB memory, assemble.sh took about 48h. On the cluster, it's been running for 72h and it is still running. I wonder if I am allocating resources correctly or if I can do something better. On my laptop, I can see in my activity monitor that canu is using many cores at the same time. Do you know how I can check this in an LSF server?

This page talks about submitting jobs on LSF. In particular, it mentions this: gridOptions="your options list" but I don't know how to use this option.

Any help would be appreciated.

|improve this question
$\endgroup$
  • 1
    $\begingroup$ It really depends, the cluster may be busy (check the queue with bjobs -u all) or maybe the reading/writing is slow if you're storing your data in the wrong locations, or it could be something else entirely $\endgroup$ – Chris_Rands Nov 24 '17 at 12:46
  • $\begingroup$ There are 125259 jobs running. My own job is running, it's not pending. Does the number of other jobs affect the running of my own job? I will ask IT if there are preferred writing locations. I did not consider that could change the speed of the job. Do you know a command to check how many resources my job is using? $\endgroup$ – charlesdarwin Nov 24 '17 at 15:42
  • $\begingroup$ That’s a lot of jobs even for a sizeable cluster, yes the jobs must be scheduled, might be easiest to ask your sys admin $\endgroup$ – Chris_Rands Nov 24 '17 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.