I'm trying to run the WDL tasks from the GTEx RNA-Seq pipeline with Cromwell using AWS Batch as a backend.

I store the STAR alignment index on an Elastic Block Store since my Cromwell server instance lacks sufficient disk space. The EBS Block is mounted to my Cromwell server at /data. The STAR index is located at /data/genomic-references/rnaseq/star_index_oh50.

Try as I might, I cannot get Cromwell to run the STAR task correctly.

My JSON inputs file:

   "rnaseq_pipeline_fastq_workflow.prefix": "s3://mybucket/test/CL100148068",
   "rnaseq_pipeline_fastq_workflow.star.fastq1": "s3://anotherbucket/genotypedata/sample1.fq.gz",
   "rnaseq_pipeline_fastq_workflow.star.star_index": "/data/genomic-references/rnaseq/star_index_oh50/",
   "rnaseq_pipeline_fastq_workflow.star.memory": "32",
   "rnaseq_pipeline_fastq_workflow.star.num_threads": "4",
   "rnaseq_pipeline_fastq_workflow.star.disk_space": "200",
   "rnaseq_pipeline_fastq_workflow.star.num_preempt": "0" 

Error message:

tar: /data/genomic-references/rnaseq/star_index_oh50/: Cannot open: No such file or directory

Note that /data/genomic-references/rnaseq/star_index_oh50/ does exist and is readable.

The relevant command in the GTEx container is here:

tar -xvvf ${star_index} -C star_index --strip-components=1

If the files are stored locally, then why does tar fail to wrap the directory?


Somebody helped me with this one through other channels, but I wanted to make it publicly available:

Make a tarball of the STAR index. The option --strip-components=1 strips the first level of the path from the untarred items; thus, the expected tarball was made with something like

tar -cvf ${star_index} the_star_index_directory/

Then point Cromwell/WDL to the tarball, e.g. on S3. The GTEx container will unwrap the tarball on the fly during execution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.