How to configure Cromwell to work with slurm?

Question

I have a cluster that is set up with slurm and I seek to use Cromwell to dispatch jobs to the slurm scheduler. I followed the description explained on Cromwells Documentation Website. And created a configuration file called cromwellconfig.conf, however when I run it using the command java -Dconfig.file=/lccs/home/logio9/Configure_Cromwell/cromwellconfig.conf \ -jar cromwell-77.jar run test-workflow.wdl I get an error parsing the config file. I am confused as to where the error is, bellow is the contents of the configuration file:

include required(classpath("application"))

backend {
  default = SLURM
  providers {
    SLURM {
      actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
      config {
        runtime-attributes = """
        Int runtime_minutes = 600
        Int cpus = 2
        Int requested_memory_mb_per_core = 8000
        String queue = "short"
        String jobname = "testwork"
        String cwd = "/lccs/home/logio9/Configure_Cromwell"
        String err = "errors.output"
        String out = "ouput.out"
        """
        
      
        submit = """
            sbatch -J ${jobname} -D ${cwd} -o ${out} -e ${err} -t ${runtime_minutes} -p ${queue} \
            ${"-c " + cpus} \
        --mem-per-cpu ${requested_memory_mb_per_core} \
            --wrap "/bin/bash ${script}"
    """  
    kill = "scancel ${job_id}"
        check-alive = "squeue -j ${job_id}"
        job-id-regex = "Submitted batch job (\\d+).*"
      }
    }
  }
}
  

This is the test wdl script I made, it copies a very large directory into another directory.

version 1.0

workflow Greetings {
    call SayHello {
    }
}

task SayHello {
    command <<<
        cp -R /lccs/home/logio9/Configure_Cromwell/tf1 /lccs/home/logio9/Configure_Cromwell/tf2

    >>>
    output {
        Array[String] out = read_lines(stdout())
    }
}

If anyone has had experience with this, please do share a bit on how exactly this configuration works.

Answer 1

Cromwell already defines the cwd, err and out runtime attributes by itself (it also defines a job_name and script). So when you define these variables in your config it causes a name conflict, since they already exist.

Cromwell will make a directory inside of the "cromwell-executions" directory (created when you run cromwell) for each job it runs. It will write the script, various stdout and stderr files, and the rc file to this directory. The intend is also for the job to run in that directory. Hence cromwell provides these runtime attributes to point to the appropriate locations within this directory so they can be used in your custom submission command.

You can try just removing these variables in the runtime-attributes section, while keeping them in the submit section.