How do I use bash for loops to loop through a WDL Array?

Question

Within the example below a WDL array of strings, test is defined as "a","b","c","d" the value total is a variable that signifies the length of the WDL array. Using the bash for loop, which will iterate from 1 up to length of the array, I seek to print out the values at each indicy in the array and output it to another array. For example when the loop is on 1 the value at indicy 1 within the array will be printed out in this case "a" and stored in the array out, if the loop is on 2 then the value "b" will be printed out and stored in the array out, etc. I tried the below example, however, I get an error when running. Can anyone help me with this problem?

How I tried to do this in WDL:

task A {
    input {
        Array[String] test = ["a","b","c","d"]
        Int total = length(test)
    }

    command {
       for (( c = 1; c <= ${total}; c++ ))
       do 
           echo ${test[$c]}
       done
    }
    output {
        Array[String] out = read_lines(stdout())
    }
}

Answer 1

I suspect the issue here is that you are trying to use a bash variable inside of a WDL placeholder. The ${} are interpreted by WDL as places where some value should should be inserted into the command. So when you say echo ${test[$c]}, test[$c] is seen a WDL expression. This isn't a valid WDL expression, causing a parsing error.

The choice to use ${} as placeholder syntax in WDL was a rather inconvenient decision by the WDL team, because it clashes with the bash syntax. There is an alternative syntax available for the command section in WDL 1.0, which is preferred for this very reason. This alternative uses ~{} as placeholder syntax, instead of ${}, and wraps the command in <<< >>> (referred to as the "heredoc-like syntax"), rather than the usual { }. In this heredoc-like syntax the ${} isn't seen as a WDL placeholder and can, thus, be used in bash commands like normal. You can read about it in the WDL SPEC.

Using this syntax your WDL would look something like this:

version 1.0

workflow test {
    call A
}

task A {
    input {
        Array[String] test = ["a","b","c","d"]
        Int total = length(test)
    }

    command <<<
       ARRAY=(~{sep=" " test}) # Load array into bash variable
       for (( c = 0; c < ~{total}; c++ )) # bash array are 0-indexed ;)
       do
           echo ${ARRAY[$c]}
       done
    >>>

    output {
        Array[String] out = read_lines(stdout())
    }
}

Note that if the values in your array are more complex and might include characters with special meaning in bash (like spaces), it would probably be better to wrap the values in single quotes like so:
ARRAY=('~{sep="' '" test}')

And just in case you weren't aware, bash for-loops can also loop over arrays directly:

command <<<
       for c in ~{sep=" " test}
       do
           echo $c
       done
>>>