How can you put a bash array into a wdl variable?

Question

Given two separate wdl arrays, my command section will do some modifications to each array and place the output into two separate bash arrays. Finally, I want to output the two separate bash arrays in the output section of the wdl task to be used in other tasks within the workflow. The below code is an example of what I would like to do:

task AddA {
    input {
        Array[String] wdllistX = [ "True", "False"]
        Array[String] wdllistY = [ "Good", "Food"]
    }

    command <<<
        ###We add the letter 'a' at the end of each string in each wdl list and finally get the 
      bash arrays:

        bashX = ("Truea" Falsea")
        bashY = ("Gooda" "Fooda")
    >>>
    output {
        Array[String] finX = bashX
        Array[String] finY = bashY
    }
}

The above example produces an error. It seems that I might have to convert the bash arrays back into wdl arrays in order to declare them as outputs in the output section, however, I am not sure how to do this. May someone help me with this? Moreover, a general explanation of how you can capture the output of specific things from the command section into the output section would be helpful aswell.

Answer 1

When a WDL task is run, the execution engine takes the command block and evaluates it as a template, rendering any wdl expressions it contains into the final script that is executed, BEFORE it is actually executed.

The actual execution environment is generally isolated from the execution engine or from other tasks. Information is passed between tasks by reading from or passing around file based outputs and not from accessing the bash environment directly, which the engine does not have access to.

So, in order to achieve what you are trying to do, you first need to get the values into a file format, and then you need to read those values using an engine function provided by WDL. Generally, all outputs need to be in a file if you want the engine to be able to interact with them. Please note that stdout and stderr are considered "files" and are accessible with stdout() and stderr() engine functions.

Option 1: Construct json files and use read_json

task AddA {
    input {
        Array[String] wdllistX = [ "True", "False"]
        Array[String] wdllistY = [ "Good", "Food"]
    }

    command <<<
        ###We add the letter 'a' at the end of each string in each wdl list and finally get the 
      bash arrays:
        echo '["Truea","Falsea"]' > x.json
        echo '["Gooda","Fooda"]' > y.json
    >>>
    output {
        Array[String] finX = read_json("x.json")
        Array[String] finY = read_json("y.json")
    }
}

Option 2: use read_lines

task AddA {
    input {
        Array[String] wdllistX = [ "True", "False"]
        Array[String] wdllistY = [ "Good", "Food"]
    }

    command <<<
        ###We add the letter 'a' at the end of each string in each wdl list and finally get the 
      bash arrays:

        echo "Truea" >> x.txt
        echo "Falsea" >> x.txt
        echo "Gooda" >> y.txt
        echo "Fooda" >> y.txt
    >>>
    output {
        Array[String] finX = read_lines("x.txt")
        Array[String] finY = read_lines("y.txt")
    }
}