Hide link, remove Update quote
Source Link
llrs
  • 4.6k
  • 1
  • 13
  • 39

Then, I tried following snakemake tutorial:

https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.htmltutorial

 . And created cluster.json based on the SLURM parameters that I need:

Update

Then, I tried following snakemake tutorial:

https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html

  And created cluster.json based on the SLURM parameters that I need:

Update

Then, I tried following snakemake tutorial. And created cluster.json based on the SLURM parameters that I need:

added 554 characters in body
Source Link
Nikita Vlasenko
  • 2.5k
  • 1
  • 17
  • 33

Update

I removed \ - line separators in the snakemake.sh script:

#!/bin/bash

snakemake -j 10 --cluster-config cluster.json --cluster "sbatch -A {cluster.A} -p {cluster.p} -t {cluster.time} --output {cluster.output} --error {cluster.error} --nodes {cluster.nodes} --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem {cluster.mem}"

And it started to run. It is not convenient though for me. I would rather prefer submitting just one job using .slurm file passing all of the parameters from #SBATCH. Is it possible?

Update

I removed \ - line separators in the snakemake.sh script:

#!/bin/bash

snakemake -j 10 --cluster-config cluster.json --cluster "sbatch -A {cluster.A} -p {cluster.p} -t {cluster.time} --output {cluster.output} --error {cluster.error} --nodes {cluster.nodes} --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem {cluster.mem}"

And it started to run. It is not convenient though for me. I would rather prefer submitting just one job using .slurm file passing all of the parameters from #SBATCH. Is it possible?

Source Link
Nikita Vlasenko
  • 2.5k
  • 1
  • 17
  • 33

Running SnakeMake on cluster

I do not understand how to specify correctly the parameters on a SLURM cluster for snakemake to use them. I tried submitting the following SLURM file, but it does not work this way and the number of cores used is only 1, not 20:

#!/bin/bash

#SBATCH -p standard
#SBATCH -A overall 
#SBATCH --time=12:00:00
#SBATCH --output=snakemake%A.out
#SBATCH --error=snakemake%A.err
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=20
#SBATCH --mem=120000

snakemake

Then, I tried following snakemake tutorial:

https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html

And created cluster.json based on the SLURM parameters that I need:

{
    "__default__" :
    {
        "A" : "overall",
        "time" : "24:00:00",
        "nodes": 1,
        "ntasks": 1,
        "cpus" : 20,
        "p" : "standard",
        "mem": 120000,
        "output": "snakemake%A.out",
        "error": "snakemake%A.err"
    }
}

And ran snakemake inside a newly created snakemake.sh script:

#!/bin/bash

snakemake -j 999 --cluster-config cluster.json --cluster "sbatch -A 
{cluster.A} -p {cluster.p} \ 
-t {cluster.time} --output {cluster.output} --error {cluster.error} -- 
nodes {cluster.nodes} \
--ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem 
{cluster.mem}"

And it is giving me an error now:

sbatch: error: Unable to open file
/bin/sh: line 1: -t: command not found Error submitting jobscript (exit code 127):

I am now completely lost at what I shoould actually do. I would prefer plain regular .slurm file submission, but how to make snakemake use them? Any suggestions would be greatly appreciated.