Handling failed blocks within scattered WDL workflow

Question

I'm trying to adapt an existing workflow to make it more robust to failures. Below is a simplified example of the workflow.

workflow main {
   File filelist="list_of_input_files.txt"
   Array[String] files=read_lines(filelist)
   scatter(file in files) {
      call task1 { input: in=file }
      if(defined(task1.out)) { call task2 { input: in=task1.out } }
      if(defined(task2.out)) { call task2 { input: in=task2.out }
   }
}

task task1 {
   File in
   String out1 = basename(in,'.txt') + '_1.txt'
   command { if(...) { ./script1.sh ${in} > ${out1} } }
   output { File? out = out1 }
   runtime { ... }
}

task task2 {
   File in
   String out2 = basename(in,'.txt') + '_2.txt'
   command { if(...) {./script2.sh ${in} > ${out2} } }
   output { File? out = out2 }
   runtime { ... }
}

task task3 {
   File in
   String out3 = basename(in,'.txt') + '_3.txt'
   command { if(...) {./script3.sh ${in} > ${out3} } }
   output { File? out = out3 }
   runtime {...}
}

with my list_of_input_files.txt looking like

firstfile.txt
secondfile.txt
thirdfile.txt
...

Obviously this is oversimplified, but I am trying to catch failed blocks by making the output of each task optional (using File?) and then having an if(defined(taskX.out)) condition to call the next task. My reasoning is that if something fails (represented by the if(...) within the command block of each task), then the out file will never be created in that task and so won't exist as input for the next task and thus if(defined(taskX.out)) will resolve as false and skip the next task (and therefore all the subsequent tasks).

With current testing, the first file in the list satisfies the if(...) conditions of all the tasks and succeeds in giving me the final output file firstfile_1_2_3.txt. However, I am getting a Required file output 'secondfile_1.txt' does not exist' error message because it fails the if(...) condition in task1 and so the output file is never created.

From what I've read in the WDL v1.1 spec, optional File outputs are allowed and are undefined if the files they point to don't exist. However, after much confused googling, I've realised it's probably because I'm using Cromwell, which is stuck with WDL v1.0.

My question is whether this should be working in WDL v1.0 and, if not, is my logic correct and should this (psuedo-)code work with WDL v1.1? Is there a way to achieve what I'm trying to do using pure WDL v1.0?

Answer 1

Optional outputs should work in WDL 1.0. Here is an example that runs without issue through cromwell (version 74).

Note that for the second call in the scatter the input file needs to be converted to from an optional to an non optional value. Since this only gets run if this value is defined, this conversion can safely be done using the select_first function (which returns the first defined value in a given list of optionals).

version 1.0

workflow test {
    input {
        File filelist = "files.txt"
    }
    Array[String] files = read_lines(filelist)

    scatter (file in  files) {
         call blah as b1 {input: in=file}
         if (defined(b1.out)) {
             call blah as b2 {input: in=select_first([b1.out])}
         }
    }
}


task blah {
    input {
        File in
    }
    String outPath = basename(in,'.txt') + '_blah.txt'

    command {
        if $(grep -q "yay" ~{in})
          then
             echo yay > ~{outPath}
        fi
    }

    output {
        File? out = outPath
    }
}

files.txt looks like this:

test1.txt
test2.txt

With test1.txt containing yay and test2.txt containing nay.