0
$\begingroup$

How could I filter outputs of a process in the input of the next process? Filtering works fine in channel, but if I try to filter outputs I got compilation error. I tried as follows:

Channel
.fromFilePairs("${params.dir}*.{bed,bim,fam}",size:3)
.set {data}
    process step1a {
    input:
    tuple val(sample_name), path(bfiles) from data
    output:
    tuple val("${prefix}"), path("${prefix}.{bed,bim,fam}") into (step1a_out,step1a_bim)
    script:
    """
    plink --bfile "${sample_name}" --chr 1-22 --out "${prefix}.step1b" --make-bed
    """
    }
    process step1b {
    input:
    tuple val(bim), path(bims) from step1a_bim.filter{bim, files -> name =~/*bim/}
    tuple val(sample_name), path(bfiles) from step1a_out
    output:
    path("${prefix}.step1b.snplist.txt")
    tuple val("${sample_name}"), path("${sample_name}.step1c.{bed,bim,fam}") into step1b_results
    script:
    """
    awk '{ if ((\\\$5=="T" && \\\$6=="A")||(\\\$5=="A" && \\\$6=="T")||(\\\$5=="C" && \\\$6=="G")||(\\\$5=="G" && \\\$6=="C")) print \\\$2, "ambig" ; else print \\\$2 ;}' "${prefix}.step1b.bim" | grep ambig > "${prefix}.step1b.snplist.txt"
    plink --bfile "${prefix}.step1b" --exclude "${prefix}.step1b.snplist.txt" --make-bed --out "${prefix}.step1c"
"""

It gives me

Compilation error
-cause: Unexpected input: '{'
process step1b {

Step1a gives me chr1.step1b.bed|bim|fam, chr2.step1b.bed|bim|fam.......chr22.step1b.bed|bim|fam (22 plink files) I need to access prefix of those outputs to run plink and other tools to generate chr1.step1c.bed|bim|fam ...chr22.step1c.bed|bim|fam (another 22 plink files). run awk to generate chr1.step1b.snplist.txt .... chr22.step1b.snplist.txt (22 txt files). Any help to filter the inputs so that I could use only *bim files from step1a_bim? Also how to select prefix of files in the input of processes that are from step1a_out to run plink? Does it make sense now?

Best Reagrds Zillur

Improve this question
$\endgroup$
2
  • $\begingroup$ In case of a compilation error it is more useful if you post your actual code, properly formatted, and your error message. $\endgroup$
    – Pallie
    Commented Mar 23, 2021 at 12:04
  • $\begingroup$ @Pallie, I edited my question above. Does it clarify my issues on outputs and inputs? $\endgroup$
    – zillur rahman
    Commented Mar 23, 2021 at 13:07

1 Answer 1

Reset to default
1
$\begingroup$

Instead of outputting all files and filtering for only bim, output only bims:

output:
tuple val("${prefix}"), path("${prefix}.{bed,bim,fam}") into (step1a_out,step1a_bim)

Do

output:
tuple val("${prefix}"), path("${prefix}.{bed,bim,fam}") into step1a_out
tuple val("${prefix}"), path("${prefix}.bim") into step1a_bim

So you can skip your complicated filter:

process step1b {
    input:
    tuple val(bim), path(bimfile) from step1a_bim
    output:
    path("${prefix}.step1b.snplist.txt")
tuple val("${sample_name}"), path("${sample_name}.step1c.{bed,bim,fam}") into 
step1b_results
    script:
    """
    awk '{ if ((\\\\\$5=="T" && \\\\\$6=="A")||(\\\\\$5=="A" && \\\\\$6=="T")||(\\\\\$5=="C" && \\\\\$6=="G")||(\\\\\$5=="G" && \\\\\$6=="C")) print \\\\\$2, "ambig" ; else print \\\\\$2 ;}' "$bimfile" | grep ambig > "${prefix}.step1b.snplist.txt"
    plink --bfile "${prefix}.step1b" --exclude "${prefix}.step1b.snplist.txt" --make-bed --out "${prefix}.step1c"

"""

```
Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks @Pallie . I tried your method. In that case, I got Process 'step1b' input file name collision -- There are multiple input files for each of the following file names: chr5.step1b.bim. Any suggestion? $\endgroup$
    – zillur rahman
    Commented Mar 23, 2021 at 17:44
  • $\begingroup$ The step1a_bim is a subset of step1a_out which produce bed|bim|fam files. $\endgroup$
    – zillur rahman
    Commented Mar 23, 2021 at 18:20
  • $\begingroup$ I could divide process step1b into two processes and could use step1a_out. But how could I use only *.bim files as input? Because step1a_out produce all *.bim, *.bed and *.fam files. For one process I need all three (bed|bim|fam) and for another I need only bim files. I am not sure how to subset the elements of a channel. $\endgroup$
    – zillur rahman
    Commented Mar 23, 2021 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.