6
$\begingroup$

It seems that in snakemake the script specified after the --jobscript cannot be used properly if a workdir: is specified in the snakefile. The path of the script specified becomes relative to the workdir defined in the snakefile instead of being relative to the current working directory. I do not know if this is a feature or a bug !!

Nevertheless, it becomes quite painful to workaround this problem. The only way I found was to parse the command line in the snakefile, for example: snakemake -c "qsub" -j 30 --js ./sge.sh --latency-wait 30 -rp in order to copy ./sge.sh in the output directory defined by the keyword workdir in the snakefile.

As a side effect, if you specify option for qsub on the command line, for example: snakemake -c "qsub -e ./logs/ -o ./logs/" -j 30 --js ./sge.sh --latency-wait 30 -rp the logs folder must be created under the workdir directory.

Is there something I don't understand with the --jobscript option? Am I not using it the right way? Am I going against the best practice?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

To quote from the snakemake documentation

All paths in the snakefile are interpreted relative to the directory snakemake is executed in. This behaviour can be overridden by specifying a workdir in the snakefile:

workdir: "path/to/workdir"

Usually, it is preferred to only set the working directory via the command line, because above directive limits the portability of Snakemake workflows.

It's not explicitly stated, but it's sort of implied that all relative paths then become relative to the working directory. I would expect that specifying an absolute path would get around this.

As an aside, in my mind setting the working directory is usually only needed on clusters without a shared file system (presumably shared between the worker nodes, but not with the head node), since there you can't cd $WORKDIR before running snakemake. This is then normally done with your scheduler, in such cases.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thx for your answer. I had seen this statement in the doc. I understand that all path in the snakefile are interpreted relative to the workdir (very practical by the way) but I feel it is still strange behavior that an option of the command line is interpreted the same way. Moreover, setting a workdir simplifies a lot the writing of the inputs and outputs (no need to prefix all files with the output dir you want) $\endgroup$
    – Eric C.
    Commented Jul 19, 2017 at 8:24
  • $\begingroup$ @EricC. I agree in this case, I would also have preferred that the working dir not apply to paths specified on the command line. $\endgroup$
    – Devon Ryan
    Commented Jul 19, 2017 at 9:54
  • 1
    $\begingroup$ yes maybe Johannes Köster will one day change this behavior, except if for some reason I don't get, it has other drawbacks. $\endgroup$
    – Eric C.
    Commented Jul 20, 2017 at 12:28
1
$\begingroup$

It's just a guess, but I think you might want to use --keep-target-files flag to not to adjust paths of targets. From snakemake manual :

–keep-target-files

Do not adjust the paths of given target files relative to the working directory.

Default: False

So if I understand right Devon's answer, combination of this flag and setting of workdir could force compute target in different directory that the snakemake is executed in.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.