Nested variable expansion in WDL code block

Question

I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter_file}” below) in the program file that will expand to include the adapter filename, like so:

Adapter file:

>IL_Multiplex_five_prime_fwd
AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
>IL_Multiplex_three_prime_rev
AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC

Program file:

ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads
MINLEN:36
SLIDINGWINDOW:4:15
LEADING:3
TRAILING:3

In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter_file} variable to expand inside the ~{trim_program_array} variable. Here’s a simplified version of my trimmomatic WDL file:

Sample workflow:

version 1.0
 
Workflow trim {
    input {
        File fastq1 = “R1.fastq.gz”
        File fastq2 = “R2.fastq.gz”
        File program_file = “trim_program.txt”
        File adapter_file = “adapters.fa”
    }
 
    Array[String] trim_program_array = read_lines(program_file)
 
    call trimmomatic {
        input:
            fastq1 = fastq1,
            fastq2 = fastq2,
            trim_program_array = trim_program_array,
            adapter_file = adaptor_file
    }
}
 
Task trimmomatic {
    input {
        File fastq1
        File fastq2
        File outfile1 = “trimmed_R1.fastq.gz”
        File outfile2 = “trimmed_R2.fastq.gz”
        Array[String] trim_program_array
        File adapter_file
     }
 
     command {
        TrimmomaticPE \
        ~{fastq1} ~{fastq2} \
        ~{outfile1} ~{outfile2} \
        ~{sep = “ “ trim_program_array}
     }
 
     output {
        File out1 = outfile1
        File out2 = outfile2
     }
}

My goal is for the first element of ~{sep = " " trim_program_array} (aka the first line of trim_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter_file} is nested inside trim_program_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?

Answer 1

Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with read_lines and won't get treated like WDL code. Since the lines from the program_file are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (~{}), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec here). So you can use optional inputs to make the different lines non-mandatory:

task trimmomatic {
    input {
        File fastq1
        File fastq2
        String outfile1 = “trimmed_R1.fastq.gz”
        String outfile2 = “trimmed_R2.fastq.gz”
        File? adapter_file
        Int? minlen
        String? slidingwindow
        Int? leading
        Int? trailing
     }
 
     command {
        TrimmomaticPE \
        ~{fastq1} ~{fastq2} \
        ~{outfile1} ~{outfile2} \
        ~{"ILLUMINACLIP:" + adapter_file + ":2:30:10:7:keepBothReads"} \
        ~{"MINLEN:" + minlen} \
        ~{"SLIDINGWINDOW:" + slidingwindow} \
        ~{"LEADING:" + leading} \
        ~{"TRAILING:" + trailing}
     }
 
     output {
        File out1 = outfile1
        File out2 = outfile2
     }
}

If, say, adapter_file were to be omitted in the inputs, then the produced command should look something like this:

TrimmomaticPE \
R1.fq R2.fq \
trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
 \
MINLEN:36 \
SLIDINGWINDOW:4:15 \
LEADING:3 \
TRAILING:3

Also note that the output file paths in the input section should be Strings. Since they don't exist yet, the execution engine will complain that it can't find them if they are Files

Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the :2:30:10:7:keepBothReads bit could be split up into additional inputs as well.