2
$\begingroup$

In Nextflow I have a process that creates a channel that holds several files in a tuple. I'm trying to call only one of those files as input for a subsequent process.

So basically if I run channel_name.view() the channel shows [/home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bestTree.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bipartitions.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bipartitionsBranchLabels.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bootstrap.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_info.tree] . I need just one file so I'm using the full name in the input section of the process.

input:
file 'RAxML_bipartitionsBranchLabels.tree' from channel_name

That does not find the file. On the other hand, if I use .flatten(), like this:

input:
file 'RAxML_bipartitionsBranchLabels.tree' from raxml_out_ch.flatten()

It works but reads some files before reaching to the one that matches and so the process results in error even after completing the task for this file properly and gives several outputs (for each file it looks into), some of them empty and one with the result.

[7c/2971f1] Submitted process > tree_visualisation (4)
[8d/dd129b] Submitted process > tree_visualisation (1)
[cb/0f91f5] Submitted process > tree_visualisation (5)
[6c/fb23c8] Submitted process > tree_visualisation (3)
ERROR ~ Error executing process > 'tree_visualisation (5)'

Caused by:
  Process `tree_visualisation (5)` terminated with an error exit status (1)

Command executed:

  Rscript tree.R RAxML_bipartitionsBranchLabels.tree phylogram.png

I'm looking for a way to filter this input so that the process only runs for this file and doesn't try with the others.

How can I do that?

$\endgroup$
3
  • 1
    $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Apr 19 at 0:23
  • 1
    $\begingroup$ Be more specific, is there a certain text pattern that this file matches? Is it always the first or the last file? Is it a file with an extension? Does the channel contain 1 tuple with multiple files or multiple file entries as separate emissions? $\endgroup$
    – Pallie
    Apr 19 at 6:59
  • $\begingroup$ That is an unusual setup, and it will likely end up filtering the channel for that specific file. Please provide a minimal reproducible example. $\endgroup$
    – user3051
    Apr 19 at 10:12

1 Answer 1

2
$\begingroup$

If, in your input definition, you specify a string to represent your filename and provide multiple file inputs as a collection, a numerical suffix is appended to each filename:

When a target file name is defined in the input parameter and a collection of files is received by the process, the file name will be appended by a numerical suffix representing its ordinal position in the list.

When you use the flatten operator, the collection is flattened such that each entry becomes a separate entry in the output channel. So for each of the five 'tree' files, five tasks are spawned: one for each 'tree' file. These files are each localized using the filename specified: 'RAxML_bipartitionsBranchLabels.tree`. This usually isn't what you want to do.

What you're looking for, I think, is some way to select the file called 'RAxML_bipartitionsBranchLabels.tree' from the initial collection of files. There are many ways to do this. For example if you wanted to flatten the output channel, it's possible to select the file you want using the first operator and a closure:

bipartitionsBranchLabels = raxml_out_ch.flatten().first {
    it.name == 'RAxML_bipartitionsBranchLabels.tree'
}

However you should only seldom need to do this. If your pipeline requires just one of these files in a downstream process, my suggestion would be to alter the upstream output declaration to include a separate output channel for this purpose:

process RAxML {

    output:
    path 'RAxML_*.tree' into raxml_out_ch
    path 'RAxML_bipartitionsBranchLabels.tree' into raxml_bipartitionsBranchLabels

    """
    touch RAxML_bestTree.tree
    touch RAxML_bipartitions.tree
    touch RAxML_bipartitionsBranchLabels.tree
    touch RAxML_bootstrap.tree
    touch RAxML_info.tree
    """
}
$\endgroup$
2
  • $\begingroup$ Thank you Steve. The last option you propose wouldn't really work with that program and I tried the others without success. What I did was to use .collect() in the input of the following process, this way it turns the outputs (of one channel) into an array (or tuple) and then I used def = ... within the script seccion to assign each file in the array to a letter. $\endgroup$
    – jose lopez
    May 26 at 22:41
  • $\begingroup$ No worries @joselopez. The last option should be tool agnostic unless I'm mistaken. It's difficult to say what the right solution is here without the details. Calling collect() seems redundant, since 'channel_name' (i.e. 'raxml_out_ch') already provides an array (java.util.ArrayList). I think you could also do a def branch_labels = yourfilelist.find { it.baseName == 'RAxML_bipartitionsBranchLabels' } in your script block to get the file you want. Note that this solution will localize all the files in the array, which may not be what you want. Your question mentioned you only needed the one. $\endgroup$
    – Steve
    May 26 at 23:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.