Skip to main content

Questions tagged [workflow-management]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

Why is my nextflow script executed sequentially?

I have the following Nextflow script: ...
MauriRamone's user avatar
1 vote
1 answer
183 views

Remove files after process is terminated

In my pipeline nextflow, I create a channel with channel.fromSRA and the channel contains lots of heavy files.fastq.gz. Then I have a first process to unzip files and transform them in files.fasta and ...
eva fonta's user avatar
2 votes
1 answer
326 views

Nextflow tuple from CSV cols

I am trying to create a channel with a CSV file and use it as tuple input. Yet the output I obtain does not contain the entire file but only the first element of the file (1st row). the csv ...
Loïc Borcard's user avatar
4 votes
1 answer
1k views

How to solve Nextflow error: "Trace file already exists"?

When trying to run epi2me-labs/wf-artic, I get the following error: ...
Cornelius Roemer's user avatar
1 vote
0 answers
62 views

Implement a WDL workflow for a docker container I built

I start saying my background is Biology and I'm totally new to WDL, so I'm aware my code will be messy but I'm trying to learn. My problem is, essentially, to run a Docker container I built and tested ...
Matteo's user avatar
  • 21
1 vote
1 answer
314 views

Handling failed blocks within scattered WDL workflow

I'm trying to adapt an existing workflow to make it more robust to failures. Below is a simplified example of the workflow. ...
tiit_helimut's user avatar
2 votes
1 answer
398 views

Mounting a folder from the host to a Docker, while running it as Cromwell runtime

I have a pipeline, which is written in WDL & runs with the help of Cromwell. I use Docker...
crusher083's user avatar
0 votes
0 answers
349 views

BWA-MEM2 alignment-Snakemake

I have started using snakemake 6.5.2 to align fastq files with reference file. I have pasted the error below in this question. How to allocate memory in the snakefile and read the header from samfile, ...
Atom's user avatar
  • 11
2 votes
1 answer
580 views

Nextflow: How to initialize channel from CSV file but only after CSV file created?

Process A runs a python script which generates a CSV file in some absolute file path say "/mnt/x/blah.csv". Process B needs to consume the CSV file line by line, after process A has finished....
Brendan Hill's user avatar
2 votes
1 answer
724 views

How to execute several processes sequentially, without channels?

Using DSL2. I have processes A, B, C which I want to run in sequence - B waits for A to finish, C waits for B to finish. They are not communicating via channels so I can't do A.out.collect(). I have ...
Brendan Hill's user avatar
1 vote
1 answer
2k views

How to trigger next process on completion of previous process?

Simple question but I am stumped. (I am still learning Nextflow) I have a process which takes as input a channel of files and processes each file separately. There is a second process which needs to ...
Brendan Hill's user avatar
2 votes
1 answer
155 views

How does cromwell and AWS batch decide which compute environment (on-demand or spot) to use?

For processing genomics workflows I make use of widdle and cromwell with an AWS back-end. I have followed all examples on the AWS cromwell tutorial. In my AWS batch console I can see there are 2 job ...
user1805473's user avatar
1 vote
1 answer
441 views

Snakemake on cluster produces empty output files

Problem: When I run my Snakefile on our cluster, all the processes complete successfully. I can see from the logs that my shell scripts are being run correctly and are claiming to complete ...
Graeme Ford's user avatar
3 votes
1 answer
574 views

Access workdir defined on command line from within Snakefile

Snakemake provides access to a workflow object within a Snakefile. This allows one to, for example, have dynamic programmatic access to the directory containing the ...
Daniel Standage's user avatar
3 votes
2 answers
1k views

Autodetect max number of cores and pass as an argument in Nextflow

I am creating a pipeline in Nextflow. One step is creating a pangenome with Roary. Roary takes threads as an argument and if no number of threads is supplied as an argument it defaults to one. Is ...
TW93's user avatar
  • 449
15 votes
1 answer
3k views

What is the difference between a Bioinformatics pipeline and workflow?

I want to understand the difference between pipeline systems and workflow engines. After reading A Review of Scalable Bioinformatics Pipelines I had a good overview of current bioinformatics ...
A.Dumas's user avatar
  • 497
5 votes
1 answer
188 views

Dealing with absence of coverage

In my snakemake workflow, I use deeptools bamCoverage (in a make_bigwig rule), then ...
bli's user avatar
  • 3,130
9 votes
4 answers
1k views

Ultimate reproducibility in R?

I'm looking for a convenient and reliable way to make an R analysis reproducible, either at different times or across collaborators. Listing the package versions or a ...
Peter's user avatar
  • 2,634
8 votes
3 answers
509 views

What are the ways to keep track of branches in the analysis?

I'm going through an RNA-seq pipeline in R/Bioconductor and want to try multiple parameters at subsequent steps, for example, running clustering with different ...
Peter's user avatar
  • 2,634