Skip to main content
18 votes
Accepted

What is the difference between a Bioinformatics pipeline and workflow?

Great question! Note that from a prescriptive standpoint, the terms pipeline and workflow don't have any strict or precise definitions. But it's still useful to take a descriptive standpoint and ...
Daniel Standage's user avatar
16 votes
Accepted

Ultimate reproducibility in R?

You're right about trying to get the R environments to be as similar as possible, as easily as possible. It's not always necessary to have the exact environment to get the same results, but it ...
James Hawley's user avatar
  • 1,384
9 votes

What are the ways to keep track of branches in the analysis?

Save the different scripts with git (seems overkill) Whoa. I did an actual double take when reading this:1 it’s the opposite of overkill. Version controlling your scripts (using Git or something ...
Konrad Rudolph's user avatar
5 votes
Accepted

How to solve Nextflow error: "Trace file already exists"?

Rotation of report files (trace, execution, timeline etc) was removed recently (since version 22.10.0 in a762ed5). To overwrite the existing trace file you will ...
Steve's user avatar
  • 3,109
5 votes
Accepted

What are the ways to keep track of branches in the analysis?

The main purpose of git is to version code, which usually means sequential improvement of the codebase. While it is possible to use branches for multiple variants of the software, permanent branches ...
Iakov Davydov's user avatar
5 votes
Accepted

Autodetect max number of cores and pass as an argument in Nextflow

I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use. However, you can try to automatize this task by using ...
plat's user avatar
  • 1,042
5 votes

Ultimate reproducibility in R?

Another option I'm trying at the moment is the combination of snakemake, conda.io and knitr. Conda is a "Package, dependency and environment management for any language" and could be used to set up ...
Sebastian Müller's user avatar
3 votes
Accepted

Dealing with absence of coverage

I'll fix the bamCoverage issue in the 2.6.0 release (it may already be fixed since I rewrite how bigWig files are made for that release). You can follow my status ...
Devon Ryan's user avatar
  • 19.6k
3 votes

Autodetect max number of cores and pass as an argument in Nextflow

Here's an example I wrote recently: ...
user324810's user avatar
  • 1,125
2 votes
Accepted

Access workdir defined on command line from within Snakefile

You could query Python for the working directory within the Snakefile, since evidently Snakemake changes the actual working directory of the process to the one specified by ...
Jesse's user avatar
  • 947
2 votes
Accepted

How to trigger next process on completion of previous process?

So from this page... https://groups.google.com/g/nextflow/c/jt77_-uApMs It is the "scatter-gather" pattern. I think the answer is to collect() the channel as follows - interested to hear if ...
Brendan Hill's user avatar
1 vote

Nextflow tuple from CSV cols

I think the problem is just that you have extra whitespace at the end of the first line in your CSV file. Either remove the whitespace manually, or use the strip ...
Steve's user avatar
  • 3,109
1 vote

Handling failed blocks within scattered WDL workflow

Optional outputs should work in WDL 1.0. Here is an example that runs without issue through cromwell (version 74). Note that for the second call in the scatter the input file needs to be converted to ...
DavyCats's user avatar
  • 507
1 vote
Accepted

Mounting a folder from the host to a Docker, while running it as Cromwell runtime

Not an expert of WDL but looking at Cromwell documentation, there is a block in the configuration of Cromwell called ...
user324810's user avatar
  • 1,125
1 vote

Nextflow: How to initialize channel from CSV file but only after CSV file created?

Assuming you are using latest version of Nextflow which support the DSL 2 language, and assuming your python process output the file blah.csv, then the easiest way to move forward will be: ...
Sam's user avatar
  • 11
1 vote

How to execute several processes sequentially, without channels?

I would guess it is not possible without channels unless you are using beforeScript and afterScript. I came up with this ...
kibubu's user avatar
  • 11
1 vote

How does cromwell and AWS batch decide which compute environment (on-demand or spot) to use?

Cromwell will send jobs to the AWS Batch queue ARN defined in the cromwell.conf file in the Cromwell server. To use Spot instances you can create an AWS Batch ...
Mark's user avatar
  • 141
1 vote

Snakemake on cluster produces empty output files

Ok, so I feel like a right PawPaw. This is NOT a snakemake problem. The issue was multi-part but it was me, python and good ol' PBS/Torque Cluster environments. Empty input files: I was using a ...
Graeme Ford's user avatar
1 vote

Ultimate reproducibility in R?

checkpoint is a system for ensuring you get packages of the same time / version. However, I confess that I haven't used it and thus can't vouch for it. My feeling is that the rest will have to be ...
agapow's user avatar
  • 788
1 vote

Ultimate reproducibility in R?

One way to achieve this is via packrat, a dependency management system developed by RStudio. Setup: 1) Create project 2) Create script in project directory to ...
Peter's user avatar
  • 2,634
1 vote

What are the ways to keep track of branches in the analysis?

I quite agree with this answer by @Konrad Rudolph. I would like to emphasis that using parametrization for your scripts is what will help you avoid multiplying branches in git. So yes, use git, but ...
bli's user avatar
  • 3,130

Only top scored, non community-wiki answers of a minimum length are eligible