Questions tagged [nextflow]
The nextflow tag has no usage guidance.
53
questions
2
votes
1
answer
29
views
Nextflow Error: failed to read header from "-"
I am trying to run my nextflow pipeline, and have gotten this error:
samtools sort: failed to read header from "-"
I'm not sure why this error is ...
- 55
3
votes
1
answer
37
views
Nextflow Pipeline: Unexpected Input "{"
I'm attempting to run a nextflow pipeline and have begun getting unexpected output for a few of the processes in my pipeline. I believe this to be a syntax error, but I am not sure where to correct it....
- 55
1
vote
1
answer
14
views
file order after nextflow groupTuple
Below is the code from nextflow site (https://www.nextflow.io/docs/edge/operator.html#grouptuple).
For the output, the first is A B C, and the second is C, A.
Is there a way to make the output like ...
- 45
3
votes
1
answer
33
views
why this nextflow scripts finished after running one sample
I have 36 samples in total in the bam_files folder, with the name like this 20230306_CH_EP_C01.md.bam
I expect the code to output all the 36 samples, one by one. But the run stopped after only running ...
- 45
1
vote
0
answers
38
views
Problem with nextflow -resume and path.fromSRA
I have a problem with nextflow -resume
Here is my script :
...
- 21
1
vote
1
answer
56
views
Remove files after process is terminated
In my pipeline nextflow, I create a channel with channel.fromSRA and the channel contains lots of heavy files.fastq.gz. Then I have a first process to unzip files and transform them in files.fasta and ...
- 21
2
votes
1
answer
33
views
Nextflow tuple from CSV cols
I am trying to create a channel with a CSV file and use it as tuple input. Yet the output I obtain does not contain the entire file but only the first element of the file (1st row).
the csv
...
- 21
3
votes
1
answer
63
views
Run nextflow pipeline multiple times depending on length of a list
This question was also asked on Stack Overflow
I have a Nextflow pipeline that runs a BUSCO search on input data, returns the output and then creates a report based on the output. One of my parameters ...
- 31
2
votes
1
answer
75
views
How to run a nextflow process for each file generated by another process separately - Input tuple does not match input set cardinality error
I have a process called FINDTAIL that generates different number of files depending on the input data. Its either 2 files or four files i.e.
...
- 333
2
votes
1
answer
88
views
How to handle output from a python script that gives different outputs depending upon the condition - Nextflow
I have created three Nextflow processes: One checks the strandedness of the fastq file and outputs it to a .txt file. Second one executes a python script with different commands depending upon the ...
- 333
2
votes
1
answer
47
views
Alignment process in Nextflow not picking up the index files from the index process
I'm using bowtie2 to build a small index. After building the index, I want to pass it to the bowtie2 alignment process. The issue is bowtie2-build outputs multiple index files with the .bt2 extension. ...
- 333
2
votes
2
answers
131
views
How does one account for both single end and paired end reads as input in a Nextflow pipeline?
Suppose you have a pipeline that takes reads as input from the user and trims them.
...
- 333
2
votes
1
answer
151
views
How to keep the basename of an input file and change its extension in Nextflow?
The following Nextflow scripts takes the $gtf as an input and outputs a .bed file. What I want is to keep just the basename of the gtf file and add the .bed extension to it. I tried declaring a ...
- 333
2
votes
1
answer
122
views
How to include only the trimmed fastq files into a nextflow channel and exclude any secondary log files created?
I have the following simple nextflow script, that takes fastq paired reads and runs them through the atria adapter trimming.
...
- 333
3
votes
1
answer
346
views
How to solve Nextflow error: "Trace file already exists"?
When trying to run epi2me-labs/wf-artic, I get the following error:
...
- 367
1
vote
1
answer
57
views
Only the first file Pair is processed from Channel.fromFilePairs()
This is my first nextflow pipeline and i am struggling with the .fromFilePairs concept of Nextflow.
I want to process paired-end read files and align them against a reference genome. I use Hisat2 so ...
- 13
3
votes
1
answer
155
views
Using SED command in Nextflow script
I was wondering if someone can help me figure out this error message. I’m at the end of my Nextflow pipeline and I want to change the header in a FASTA file and it works when I use this command:
...
- 453
1
vote
1
answer
126
views
Nextflow with Minimap2 not working
I'm trying to create a Nextflow script that maps assemblies to reference gene sequences. This is what I have so far:
...
- 453
2
votes
1
answer
142
views
recover files after groupTuple
I have a process that emits BAM files with a key containing the sample name:
...
- 242
2
votes
1
answer
340
views
How can I call one file from a channel that's composed of several files in Nextflow?
In Nextflow I have a process that creates a channel that holds several files in a tuple. I'm trying to call only one of those files as input for a subsequent process.
So basically if I run ...
- 21
2
votes
1
answer
280
views
Nextflow: cannot get file pairs from Channel.fromFilePairs
I'm remarkably still struggling with some basic Nextflow concepts. I'm simply trying to collect all pairs of .cram and ...
- 1,274
3
votes
1
answer
370
views
Nextflow scheduler job doesn't end upon successful workflow completion
Bug fixed
This issue appears to have been fixed in latest Nextflow: https://github.com/nextflow-io/nextflow/issues/2693
I've been using ...
- 1,274
4
votes
1
answer
275
views
BWA mem | samtools view: Intermittent parsing error
Update
The issue was that bwa was running out of memory and failing, but that error wasn't floating to the top (see @Steve's answer, below). I was getting an error ...
- 1,274
6
votes
1
answer
478
views
Nextflow: dynamically setting resource directives for a process
I'm trying to test variations of CPU and memory for a specific process (samtools sort) to measure computational efficiency (measured only by time). To do this, I'd ...
- 1,274
4
votes
2
answers
1k
views
Nextflow (DSL v2): how to best synchronize multiple outputs from a single process
I have a workflow that needs to:
Generate .fastq files from .bams (while preserving the @RG ...
- 1,274
1
vote
2
answers
881
views
Nextflow: how to create many-to-many tuple for process
Using Nextflow, I need to submit x * y jobs where x is the number of input ...
- 1,274
1
vote
1
answer
210
views
How fo force nextflow to repeat a process until all values in a particular channels are used up BUT a single value from another channel is needed
I am trying to find a hack for getting a process to run until all emissions in a channel are used up. The problem is that the process also takes as input a second channel that only has a single ...
- 31
0
votes
1
answer
255
views
Nextflow HISAT2 command exit status 255 with prebuild index
I have a problem with my nextflow pipeline.
My workflow looks like this:
Uncompress the hisat2 genotpye index grch38_genome.tar.gz from https://genome-idx.s3.amazonaws.com/hisat/grch38_genome.tar.gz
...
1
vote
2
answers
379
views
How do I call variants using BCFTools without piping mpileup file?
I need to call variants on a large number of reads using both VarScan2 and BCFTools. Both of these variant calling tools use <...
- 123
1
vote
1
answer
858
views
Why are my Nextflow processes not executing in parallel?
I have written a Nextflow script with three process:
The first process takes a pair of fastq files and aligns with reference genome. The process writes the ...
- 123
2
votes
1
answer
383
views
Nextflow: How to initialize channel from CSV file but only after CSV file created?
Process A runs a python script which generates a CSV file in some absolute file path say "/mnt/x/blah.csv".
Process B needs to consume the CSV file line by line, after process A has finished....
- 183
3
votes
2
answers
374
views
2
votes
1
answer
415
views
How to execute several processes sequentially, without channels?
Using DSL2.
I have processes A, B, C which I want to run in sequence - B waits for A to finish, C waits for B to finish.
They are not communicating via channels so I can't do A.out.collect().
I have ...
- 183
2
votes
1
answer
496
views
splitCsv map with text, join with another channel, nextflow
I have a csv file which describes number of chunks per chromosome, like;
...
- 247
2
votes
2
answers
1k
views
Use conditional in workflow in Nextflow dsl2
How should I do:
...
if ($(params.aligner) == "bowtie") {
align_bowtie2(get_fq_info.out.fq_info)
}
else {
align_bwa(get_fq_info.out.fq_info)
}
...
...
- 557
1
vote
2
answers
864
views
Nextflow: Can I parameterize similar processes inside a for loop?
I have set of ~5 processes which perform very similar tasks such that their differences can be parameterized. Rather than putting 5 items through some channel (which seems complicated), or defining 5 ...
- 183
0
votes
0
answers
495
views
parse csv/text file to create a channel with array in nextflow
[![example csv file][1]][1]
First I want to start by parsing and doing some operation on a text/csv file. How could I read a csv/text file in nextflow?
Suppose, I have above csv file with 3 columns. ...
- 247
1
vote
1
answer
1k
views
How to trigger next process on completion of previous process?
Simple question but I am stumped. (I am still learning Nextflow)
I have a process which takes as input a channel of files and processes each file separately.
There is a second process which needs to ...
- 183
1
vote
1
answer
2k
views
Merge multiple channels each emits multiple files in nextflow
I need to merge multiple channels each emits multiple file. So far I did something as below:
...
- 247
1
vote
1
answer
313
views
Input from multiple channel, merge and use in nextflow
I need to merge multiple channel to reuse them in two processes. So far I tried as follows:
...
- 247
1
vote
1
answer
1k
views
Nextflow: run sbatch (slurm job script) inside a process
I was trying to run a slurm job script inside nextflow process. Is it possible? For instance, I have a job script 'PRE_PHASING_QC.JOB' and outside nextflow it runs perfectly, I want to integrate it in ...
- 247
1
vote
1
answer
415
views
Nextflow: Runs without error, produce few but not all output files into output channel
I was trying to develop a pipeline using nextflow and plink. My code runs without any error, but got only two output files instead of 22 (one for each chromosome). Here is part my code block, any help?...
- 247
0
votes
1
answer
573
views
nextflow: Filter outputs of a process
How could I filter outputs of a process in the input of the next process? Filtering works fine in channel, but if I try to filter outputs I got compilation error. I tried as follows:
...
- 247
1
vote
1
answer
843
views
Nextflow: ordering the execution of processes; outputs of process A are inputs of process B
I need to build a pipeline where 2nd process starts after completion of the 1st process and the 3rd after 2nd. As I understand nextflow's processes run parallel. But I need it to run in a order 1st>...
- 247
2
votes
1
answer
525
views
Nextflow: No signature of Method: chr() is applicable for argument types; unknown variable
I was trying to use plink in nextflow. My first process will split the input (single bed|bim|fam file) into 22 chromosomes (each with bed|bim|fam files) which would be used in subsequent processes. My ...
- 247
1
vote
1
answer
670
views
Nextflow error: can not invoke method minus () on null object
Hi I was trying to use plink in nextflow and getting the error mentioned in the subject. Here is my code, any help what I am missing. I am new in nextflow, it was ok when input was a single file, but ...
- 247
1
vote
1
answer
143
views
add task to nextflow and resume
in test.nf (dsl2) I have
...
workflow{
a = Channel...
processA(a)
processB(processA.out.emitA)
processC(processB.out.emitB)
}
...
- 557
1
vote
2
answers
592
views
How to merge back the processes by sample after splitting by chromosome in Nextflow?
I have a pipeline that has an input list of samples. I would like to split those samples by chromosomes, process each chromosome apart, and then merge them back by sample name.
Here's an example:
<...
- 1,045
1
vote
1
answer
103
views
Master of Pores Installation Mac OS Mojave 10.14
I'm bit of a newbie with ONT analyses and am having some trouble with the Master Of Pores installation instructions (https://biocorecrg.github.io/master_of_pores/install.html). I've managed to install ...
- 11
0
votes
1
answer
95
views
nextflow `fromPath` not passing all files with *
main.nf is
#!/usr/bin/env nextflow
params.input = ''
tchan = Channel.fromFile(params.input)
tchan.println()
I have a.bam, b.bam and c.bam
I run:
...
- 557