1
$\begingroup$

Background I am using snakemake and I want to add the external (cluster) jobid to my cluster output and error log files. Below is my shell script.

snakemake --configfiles config/config_resources.yml config/config_parameters.yml --latency-wait 60 --use-conda --printshellcmds --jobname PipeName_{name}.jobid{jobid} --jobs 300 --cluster "bsub -W 480 -n {threads} -M {resources.mem_gb}G -R 'rusage[mem={resources.mem_gb}G]' -o logs/{name}_{jobid}.out -e logs/{name}_{jobid}.err".

But I only get the internal one in de log name from snakemake (e.g. 1 to 5), instead of the one from the cluster.

Question How to I get the external job id in there and not the local one made by snakemake? Or where in my shell script did I go wrong?

Cheers and thanks in advance.

$\endgroup$

1 Answer 1

1
$\begingroup$

bsub supports some filename placeholders using a percent sign, like %J for the LSF job ID, so you can use that:

snakemake ... --cluster "bsub ... -o logs/{name}_%J.out -e logs/{name}_%J.err"

That way snakemake will format the cluster command string with its wildcards (like name) as it constructs the bsub command, and then bsub will put the job ID in place in those filenames. Since Snakemake just sees that as one big command to run for the cluster and doesn't know the details other than formatting the text, I think the two shouldn't clash.

$\endgroup$
1
  • 1
    $\begingroup$ Oh my, thank you! That worked! $\endgroup$
    – Dandelion
    Oct 14, 2022 at 16:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.