2
$\begingroup$

Within the example below a WDL array of strings, test is defined as "a","b","c","d" the value total is a variable that signifies the length of the WDL array. Using the bash for loop, which will iterate from 1 up to length of the array, I seek to print out the values at each indicy in the array and output it to another array. For example when the loop is on 1 the value at indicy 1 within the array will be printed out in this case "a" and stored in the array out, if the loop is on 2 then the value "b" will be printed out and stored in the array out, etc. I tried the below example, however, I get an error when running. Can anyone help me with this problem?

How I tried to do this in WDL:

task A {
    input {
        Array[String] test = ["a","b","c","d"]
        Int total = length(test)
    }

    command {
       for (( c = 1; c <= ${total}; c++ ))
       do 
           echo ${test[$c]}
       done
    }
    output {
        Array[String] out = read_lines(stdout())
    }
}
$\endgroup$
0

1 Answer 1

3
$\begingroup$

I suspect the issue here is that you are trying to use a bash variable inside of a WDL placeholder. The ${} are interpreted by WDL as places where some value should should be inserted into the command. So when you say echo ${test[$c]}, test[$c] is seen a WDL expression. This isn't a valid WDL expression, causing a parsing error.

The choice to use ${} as placeholder syntax in WDL was a rather inconvenient decision by the WDL team, because it clashes with the bash syntax. There is an alternative syntax available for the command section in WDL 1.0, which is preferred for this very reason. This alternative uses ~{} as placeholder syntax, instead of ${}, and wraps the command in <<< >>> (referred to as the "heredoc-like syntax"), rather than the usual { }. In this heredoc-like syntax the ${} isn't seen as a WDL placeholder and can, thus, be used in bash commands like normal. You can read about it in the WDL SPEC.

Using this syntax your WDL would look something like this:

version 1.0

workflow test {
    call A
}

task A {
    input {
        Array[String] test = ["a","b","c","d"]
        Int total = length(test)
    }

    command <<<
       ARRAY=(~{sep=" " test}) # Load array into bash variable
       for (( c = 0; c < ~{total}; c++ )) # bash array are 0-indexed ;)
       do
           echo ${ARRAY[$c]}
       done
    >>>

    output {
        Array[String] out = read_lines(stdout())
    }
}

Note that if the values in your array are more complex and might include characters with special meaning in bash (like spaces), it would probably be better to wrap the values in single quotes like so:
ARRAY=('~{sep="' '" test}')

And just in case you weren't aware, bash for-loops can also loop over arrays directly:

command <<<
       for c in ~{sep=" " test}
       do
           echo $c
       done
>>>
$\endgroup$
3
  • $\begingroup$ This is a comprehensive answer, giving a clear insight into WDL and its integration with bash. $\endgroup$
    – M__
    Mar 24, 2022 at 15:33
  • $\begingroup$ Thank you for this. In the first answer you gave, is it possible to load the array into a bash variable i.e. ARRAY=(~{sep=" " test}) within the workflow definition so it would like the bellow example? Can I then take that as an input into my task definition? workflow test { Array[String] test = ["a","b","c","d"] Int total = length(test) ARRAY=(~{sep=" " test}) call A { inputs: testarray = ARRAY } } task A { inputs { Array[String] = testarray } } $\endgroup$ Mar 27, 2022 at 20:52
  • $\begingroup$ Currently, WDL doesn't have a way of defining environment variables in WDL itself. You have to do so in the command section. There has been some discussion on introducing such a feature though: github.com/openwdl/wdl/discussions/458 $\endgroup$
    – DavyCats
    Mar 29, 2022 at 8:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.