How can I call one file from a channel that's composed of several files in Nextflow?

Question

In Nextflow I have a process that creates a channel that holds several files in a tuple. I'm trying to call only one of those files as input for a subsequent process.

So basically if I run channel_name.view() the channel shows [/home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bestTree.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bipartitions.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bipartitionsBranchLabels.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bootstrap.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_info.tree] . I need just one file so I'm using the full name in the input section of the process.

input:
file 'RAxML_bipartitionsBranchLabels.tree' from channel_name

That does not find the file. On the other hand, if I use .flatten(), like this:

input:
file 'RAxML_bipartitionsBranchLabels.tree' from raxml_out_ch.flatten()

It works but reads some files before reaching to the one that matches and so the process results in error even after completing the task for this file properly and gives several outputs (for each file it looks into), some of them empty and one with the result.

[7c/2971f1] Submitted process > tree_visualisation (4)
[8d/dd129b] Submitted process > tree_visualisation (1)
[cb/0f91f5] Submitted process > tree_visualisation (5)
[6c/fb23c8] Submitted process > tree_visualisation (3)
ERROR ~ Error executing process > 'tree_visualisation (5)'

Caused by:
  Process `tree_visualisation (5)` terminated with an error exit status (1)

Command executed:

  Rscript tree.R RAxML_bipartitionsBranchLabels.tree phylogram.png

I'm looking for a way to filter this input so that the process only runs for this file and doesn't try with the others.

How can I do that?

Answer 1

If, in your input definition, you specify a string to represent your filename and provide multiple file inputs as a collection, a numerical suffix is appended to each filename:

When a target file name is defined in the input parameter and a collection of files is received by the process, the file name will be appended by a numerical suffix representing its ordinal position in the list.

When you use the flatten operator, the collection is flattened such that each entry becomes a separate entry in the output channel. So for each of the five 'tree' files, five tasks are spawned: one for each 'tree' file. These files are each localized using the filename specified: 'RAxML_bipartitionsBranchLabels.tree`. This usually isn't what you want to do.

What you're looking for, I think, is some way to select the file called 'RAxML_bipartitionsBranchLabels.tree' from the initial collection of files. There are many ways to do this. For example if you wanted to flatten the output channel, it's possible to select the file you want using the first operator and a closure:

bipartitionsBranchLabels = raxml_out_ch.flatten().first {
    it.name == 'RAxML_bipartitionsBranchLabels.tree'
}

However you should only seldom need to do this. If your pipeline requires just one of these files in a downstream process, my suggestion would be to alter the upstream output declaration to include a separate output channel for this purpose:

process RAxML {

    output:
    path 'RAxML_*.tree' into raxml_out_ch
    path 'RAxML_bipartitionsBranchLabels.tree' into raxml_bipartitionsBranchLabels

    """
    touch RAxML_bestTree.tree
    touch RAxML_bipartitions.tree
    touch RAxML_bipartitionsBranchLabels.tree
    touch RAxML_bootstrap.tree
    touch RAxML_info.tree
    """
}