I have a pipeline that has an input list of samples. I would like to split those samples by chromosomes, process each chromosome apart, and then merge them back by sample name.
Here's an example:
samplesList = Channel.from(["sample1", "sample2"])
// A process that splits the samples by chromosome as an example
// In my actual case, I have a VCF that is split by `snpSift Split` command
// and generates files like [sample]_[chr].vcf
// In the example below, I only gave sample1
// But take into consideration that I have multiple samples
process bar {
echo true
output:
file("*.tsv") into bar_output
script:
"""
echo 1 > sample1_chr1.tsv
echo 2 > sample1_chr2.tsv
echo 3 > sample1_chr3.tsv
echo 4 > sample1_chr4.tsv
echo 5 > sample1_chr5.tsv
"""
}
// Process each chromosome apart for each sample
// This means the output of the process foo
// is only ONE chromosome per sample
process foo {
input:
each x from bar_output // to treat each file separately from the channel
output:
file("*.tsv") into xasxa
script:
"""
echo \\\$(cat $x)\\\$(cat $x) > sample1_chr\\\$(cat $x)_2.tsv
"""
}
// How should I write the next process to group back them together by sample name?
// I do not wish to do a collect() on channel foo because it will force
// all the processes of foo to finish first.
// In this example, foo would emit 5 different channels of chromosome for sample1
// This process should merge sample1 only when all chromosomes are processed
// independently of other samples.
process mergeChromosomesBySamples {
input:
???
}
As the example above, the process mergeChromosomesBySamples
should return something similar/close to
["sample1", ["/path/to/workdir/sample1_chr1.tsv",
"/path/to/workdir/sample1_chr2.tsv",
"/path/to/workdir/sample1_chr3.tsv",
"/path/to/workdir/sample1_chr4.tsv",
"/path/to/workdir/sample1_chr5.tsv"]
]
EDIT1: more explication for the process mergeChromosomesBySamples
is given in the comments and as follows ...
The channel xasxa
processes each chromosome apart, this means that the output of the channel xasxa
contains only one chromosome of the sample like "/path/to/workdir/sample1_chr1.tsv"
only. My goal is then to merge the processes out xasxa
for each sample together.
I do not wish to use collect()
in this case because it would have to force all the processes of foo to finish before. I want them to work independently for each sample.
Where each sample has its own VCF generated and I process them independently.
EDIT2: Here's a graph of what I am trying to achieve:
If I use .collect()
, then all the processing on the chromosomes of all the samples has to finish first before being able to merge them, which is something I do not want.
I appreciate your help.