2
$\begingroup$

I'm remarkably still struggling with some basic Nextflow concepts. I'm simply trying to collect all pairs of .cram and .crai files from a directory using Channel.fromFilePairs(...), but I get no output when I try to view() the Channel.

Question: Why does my Channel.fromFilePairs(...) command not return file pairs, but I get a list of files when using Channel.fromPath(...)?

As far as I can tell, I'm following the same pattern from the following examples:

  1. Nextflow fromFilePairs
  2. .bam/.bai example (except without the extra code)

Code

nextflow.enable.dsl=2

workflow {

    /*
     * Channel.fromPath returns the expected files and prints them to
     * terminal.
     */
    Channel.fromPath('/path/to/adsp/cram/snd10000/A-CUHS-CU000*.cram')
        | view()
 
    /*
     * Channel.fromFilePairs does not print anything to terminal.
     */
    Channel.fromFilePairs('/path/to/adsp/cram/snd10000/A-CUHS-CU000*.{cram,crai}')
        | view()
}

Output (note, no file tuples/pairs)

N E X T F L O W  ~  version 21.04.3
Launching `pairs.nf` [pensive_plateau] - revision: f917a5c41b
/path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram.crai
/path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram.crai
/path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram.crai
/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram
/path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram
/path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram
/path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram
/path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram.crai
/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram.crai
/path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram
$\endgroup$

1 Answer 1

3
$\begingroup$

TLDR, use:

nextflow.enable.dsl=2

workflow {

    Channel.fromFilePairs('/path/to/adsp/cram/snd10000/A-CUHS-CU000*.cram{,.crai}')
        | view()
}

Results:

N E X T F L O W  ~  version 20.10.0
Launching `xscript.nf` [special_caravaggio] - revision: a4e53657a0
[A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram.crai]]
[A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram.crai]]
[A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram.crai]]
[A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram.crai]]
[A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram.crai]]


When using the fromFilePairs factory method, the trick is to ensure that the pattern you provide can return a common prefix in order for the files to be grouped as expected. An easy way to check to see if this is working as expected is to directly call the readPrefix helper function. For example:

test_cram = file('/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram')
test_crai = file('/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram.crai')

pattern = '/path/to/adsp/cram/snd10000/A-CUHS-CU000*.{cram,crai}'

println("My CRAM prefix is: ${Channel.readPrefix(test_cram, pattern)}")
println("My CRAI prefix is: ${Channel.readPrefix(test_crai, pattern)}")

You can see that these results do not produce a common prefix:

N E X T F L O W  ~  version 20.10.0
Launching `script.nf` [lonely_jang] - revision: 428df7e55e
My CRAM prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1
My CRAI prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram

But if you change your pattern definition to:

pattern = '/path/to/adsp/cram/snd10000/A-CUHS-CU000*.cram{,.crai}'

We can check that it returns a common prefix:

N E X T F L O W  ~  version 20.10.0
Launching `script.nf` [maniac_volta] - revision: a8acf7b03f
My CRAM prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1
My CRAI prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1
$\endgroup$
1
  • 1
    $\begingroup$ Thanks, @Steve. Very helpful. The Channel.readPrefix(...) is a nice test. $\endgroup$ Mar 16, 2022 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.