Skip to main content

Questions tagged [wdl]

Workflow Description Language; used to specify data processing and analysis workflows with a human-readable and -writeable syntax.

Filter by
Sorted by
Tagged with
0 votes
0 answers
25 views

Unable to access error logs of failed cromwell execution (WDL) in local machine running in ubuntu environment

I was trying to run bases2fastq software downloaded from the element bioscience platform. I was running this software through WDL workflow named list_fastq. For some reason, maybe syntax, my cromwell ...
Jivesh Singh's user avatar
1 vote
0 answers
31 views

WDL dependencies

I thought that when you linked tasks in a WDL workflow e.g. call task1 { input: in = file1 } call task2 { input: in = task1.out } That the first time you ...
flimbar's user avatar
  • 11
3 votes
0 answers
31 views

cromwell files downloaded from a URL have too strict permissions

Cromwell can download files that are URLs (i.e. URLs in the inputs.json). However, the permissions of these files, once they are downloaded, are very strict (...
Jeff Froula's user avatar
2 votes
0 answers
80 views

cromwell fails to download url input file

I have an input file that is a URL. Cromwell (v84) fails to pull it when I'm not using "Local" as a backend. The only difference between a working case and failed case is the name of the ...
Jeff Froula's user avatar
1 vote
0 answers
92 views

Feeding the output of one task into another in WDL

Hi there I'm new to WDL and I need to complete a two-step workflow, Basically, what I need to do is to extract and merge two reads dataset (forward and reverse) for a sample and pass the merged, ...
Matteo's user avatar
  • 21
1 vote
0 answers
62 views

Implement a WDL workflow for a docker container I built

I start saying my background is Biology and I'm totally new to WDL, so I'm aware my code will be messy but I'm trying to learn. My problem is, essentially, to run a Docker container I built and tested ...
Matteo's user avatar
  • 21
1 vote
1 answer
131 views

Nested variable expansion in WDL code block

I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and ...
user16196's user avatar
2 votes
0 answers
69 views

Pulling Multiple Files via rsync with WDL

I'm trying to run a simple task to pull multiple files via rsync with WDL. The task should be quite simple but how to pull the multiple files subsequently is giving me issues. Below is a snippet of my ...
Biohacker's user avatar
  • 121
2 votes
1 answer
518 views

How can you put a bash array into a wdl variable?

Given two separate wdl arrays, my command section will do some modifications to each array and place the output into two separate bash arrays. Finally, I want to output the two separate bash arrays in ...
Logios James's user avatar
1 vote
0 answers
42 views

How to use EPAM Cloud Pipelines?

I am searching for an easy-to-use GUI management tool to handle my WDL workflows, I found EPAM Cloud Pipelines. Unfortunately, it uses cloud services such as AWS, GCP, etc. but I wanted to use my own ...
Logios James's user avatar
1 vote
0 answers
30 views

Cromwell - controlling core/mem usage, and CallCaching, while running locally?

I'm sitting on a mountain of microarray (.idat files) data, and have been tasked to analyze it. I chose the MoChA pipeline, developed by the Broad Institute. It runs through Cromwell, of which I know ...
Joel's user avatar
  • 11
0 votes
1 answer
577 views

How to configure Cromwell to work with slurm?

I have a cluster that is set up with slurm and I seek to use Cromwell to dispatch jobs to the slurm scheduler. I followed the description explained on Cromwells Documentation Website. And created a ...
Logios James's user avatar
2 votes
1 answer
867 views

How do I use bash for loops to loop through a WDL Array?

Within the example below a WDL array of strings, test is defined as "a","b","c","d" the ...
Logios James's user avatar
3 votes
2 answers
467 views

Wrapping R scripts in WDL and run in docker

I am trying to run an R script as a task in WDL in a docker image. Since this is my first time working with these, I am stuck in something very simple (and in the initial stage) and am hoping somebody ...
binn's user avatar
  • 73
1 vote
1 answer
315 views

Handling failed blocks within scattered WDL workflow

I'm trying to adapt an existing workflow to make it more robust to failures. Below is a simplified example of the workflow. ...
tiit_helimut's user avatar
2 votes
1 answer
398 views

Mounting a folder from the host to a Docker, while running it as Cromwell runtime

I have a pipeline, which is written in WDL & runs with the help of Cromwell. I use Docker...
crusher083's user avatar
0 votes
1 answer
445 views

Error when trying to move multiple file types in wdl

I tried to add a "gather_results" task to a workflow that takes in an array of Files and moves output files to a new folder of a specified name. ...
Henry C Wong's user avatar
1 vote
1 answer
438 views

Extracting WDL map keys as a task

I want to get the array of keys from a map data structure, so I can perform array-based functions against the map in subsequent tasks. I am limited to draft-2 ...
Xophmeister's user avatar
0 votes
1 answer
144 views

How do I keep a glob from copying a file in Cromwell / WDL

Using Crowell 52. When I do a glob to get matching files, Cromwell places a copy of each matching file in the glob-* directory. As these are multi-gigabyte Bam files, I'd really prefer for Cromwell ...
Greg Dougherty's user avatar
1 vote
2 answers
779 views

How do I force Cromwell / WDL to put input files in the same directory?

I have a cromwell task that, among other variables, takes these three variables: ...
Greg Dougherty's user avatar
1 vote
1 answer
619 views

What changed wwith Python in WDL change from draft 2 to version 1?

I'm updating my WDL from draft 2 (whichs works fine) to version 1. I'm stuck with a Py snippet ...
Liam McIntyre's user avatar
2 votes
1 answer
683 views

How do I rename output folder created using WDL's glob function, with cromwell execution engine?

I made a wdl (genomics pipeline language by the Broad Institute) task for fastqc that outputs the zip files, of which there will be as many as reads that the sample I'm working on at that moment has. ...
Mar's user avatar
  • 41
1 vote
0 answers
343 views

Reading nested Map data structures in WDL

I'm having trouble getting WDL's read_json to work as expected when trying to create a Map[String, Map[String, String]] data ...
bioin4's user avatar
  • 111
0 votes
1 answer
105 views

Pointing Cromwell to local EBS mounted storage on AWS Batch

I'm trying to run the WDL tasks from the GTEx RNA-Seq pipeline with Cromwell using AWS Batch as a backend. I store the STAR alignment index on an Elastic Block Store since my Cromwell server instance ...
Kevin L. Keys's user avatar
0 votes
0 answers
265 views

How do I get cromwell to create soft-links?

The file system I'm running on doesn't allow the creation of hard-links. Cromwell is trying to create hard-links to directories, and when that fails, it's copying them. Which is leading to disk ...
Greg Dougherty's user avatar
0 votes
2 answers
564 views

Scatter tasks around predefined batches inside a WDL workflow

Suppose I have two arrays Array[File] X and Array[String] Y both of length N, with Y being a sort of R-like factor that defines batch membership for X, for a total of M batches with M < N. A valid ...
Giulio Genovese's user avatar
-1 votes
1 answer
603 views

output Array[Pair[X,Y]] to file inside a WDL task

I have a table containing different metadata information about a set of samples as an Array[Object] and I need to extract a couple of columns from this table and output those two columns into a file ...
Giulio Genovese's user avatar
1 vote
1 answer
320 views

Matching files by chromosome during wdl scatter gather

I'm trying to implement a polygenic score pipeline in WDL and I'm quite new to this pipeline management. The first step of the pipeline takes a file of GWAS summary statistics and splits it by ...
Vivek's user avatar
  • 393
1 vote
1 answer
802 views

How to generate the json files for Cromwell workflow execution?

I am starting to use Cromwell WDL scripts for running GATK workflows on a slurm cluster. When using the paired-fastq-to-unmapped-bam script I need to define the fastq sepcifics in the appropriate json ...
Kristian Unger's user avatar
1 vote
1 answer
370 views

Using Broad Institute RNAseq pipeline

There is a Github report for Broad's pipeline on RNA-Seq. It has a docker build file and WDL scripts. However, I don't see how any of the WDL script is being used in the docker (Dockerfile). It looks ...
SmallChess's user avatar
  • 2,699