1
$\begingroup$

Hi I was trying to use plink in nextflow and getting the error mentioned in the subject. Here is my code, any help what I am missing. I am new in nextflow, it was ok when input was a single file, but for grouped files, I am getting lost.

params.dir='data/'
params.publish="output/"
Channel
.fromFilePairs("${params.dir}*.{bed,fam,bim}",size:3){file -> file.baseName}
.set {plink-data}
    
process test1{
publishDir params.publish
input:
set t1,file(plk1) from plink-data
output:
file "${t1}.{bed,fam,bim}" into test1_results
script:
output_file = "${t1}"
base = plk1[0].baseName
"""
module load plink
plink --bfile $t1 --maf 0.05 --make-bed --out ${output_file}
"""
}
process test2{
publishDir params.publish
input:
set file(bed), file(fam), file(bim) from test1_results.collect()
output:
set t2,file '${plk2}.{bed,fam,bim}' into test2_results
script:
output_file = "{t2}"
base = plk2[0].baseName
"""
plink --bfile $t2 --maf 0.01 --make-bed --out ${output_file}
"""
}
$\endgroup$
2
$\begingroup$

Channel names must consist only alphanumeric characters including underscores. Your channel called 'plink-data' contains a hyphen. This is problematic because it is interpreted as a minus() operation to remove 'data' from 'plink'. You get a NullPointerException since 'plink' hasn't been defined. You can test this easily with the following example:

Channel
    .of(1..23, 'X', 'Y')
    .set { chr-names }

chr-names.view()

Results:

Cannot invoke method minus() on null object

The solution is to just use an underscore instead of a hyphen:

Channel
    .of(1..23, 'X', 'Y')
    .set { chr_names }

chr_names.view()

Follow-up issues:

I think if you want to use the bfiles plink option (and I think this is reasonable), I think it makes the most sense to just keep your bfiles in a tuple:

Channel
    .fromFilePairs( "${params.input_dir}/*.{bed,bim,fam}", size: 3 )
    .set { plink_data }


process test1{

    publishDir params.publish_dir

    module 'plink'

    input:
    tuple val(sample_name), path(bfiles) from plink_data

    output:
    tuple val("${sample_name}-t1"), path("${sample_name}-t1.{bed,bim,fam}") into test1_results

    """
    plink \\
        --bfile "${sample_name}" \\
        --maf 0.05 \\
        --make-bed \\
        --out "${sample_name}-t1"
    """
}

process test2{

    publishDir params.publish_dir

    module 'plink'

    input:
    tuple val(sample_name), path(bfiles) from test1_results

    output:
    tuple val("${sample_name}-t2"), path("${sample_name}-t2.{bed,bim,fam}") into test2_results

    """
    plink \\
        --bfile "${sample_name}" \\
        --maf 0.01 \\
        --make-bed \\
        --out "${sample_name}-t2"
    """
}

If you need to reference each bfile separately, you could just unpack the tuple and reference each file accordingly:

Channel
    .fromFilePairs( "${params.input_dir}/*.{bed,bim,fam}", size: 3 )
    .map { group_key, files -> tuple( group_key, *files ) }
    .set { plink_data }


process test1{

    publishDir params.publish_dir

    module 'plink'

    input:
    tuple val(sample_name), path(bed), path(bim), path(fam) from plink_data

    output:
    tuple val("${sample_name}-t1"), path("${sample_name}-t1.{bed,bim,fam}") into test1_results

    """
    plink \\
        --bed "${bed}" \\
        --bim "${bim}" \\
        --fam "${fam}" \\
        --maf 0.05 \\
        --make-bed \\
        --out "${sample_name}-t1"
    """
}

process test2{

    publishDir params.publish_dir

    module 'plink'

    input:
    tuple val(sample_name), path(bed), path(bim), path(fam) from test1_results

    output:
    tuple val("${sample_name}-t2"), path("${sample_name}-t2.{bed,bim,fam}") into test2_results

    """
    plink \\
        --bed "${bed}" \\
        --bim "${bim}" \\
        --fam "${fam}" \\
        --maf 0.01 \\
        --make-bed \\
        --out "${sample_name}-t2"
    """
}

Note that in the first example, bfiles could be replaced with "*" since no reference is made to the tuple of bfiles anyway. Also note that the list of files produced using the fromFilesPairs operator are returned in lexicographic order.

$\endgroup$
10
  • 1
    $\begingroup$ No worries @zillurrahman, I have added some example code to show how you could declare your inputs and outputs. The inputs could just be a tuple of bfiles or each file could be referenced separately. I haven't used plink before and have only had a (very) quick read of the docs, so there may be some more issues. Hopefully it's enough to get you started. $\endgroup$
    – Steve
    Mar 20 '21 at 3:28
  • 1
    $\begingroup$ Thanks a lot @Steve. It works as I expected! One thing, plink also generate a log file, I need this as one of the outputs but the log will not be used in the next process. Now I am adding more processes in the script, I will get back to you if I got stuck again. $\endgroup$ Mar 20 '21 at 9:35
  • 1
    $\begingroup$ So for Nextflow to be able to publish a file using the publishDir directive, it (the file) needs to be referenced in your process' output declaration block somehow. If it's not required by any downstream process, you can just add another output channel, for example: path("${sample_name.log}") into test1_logs. Please let me know if that doesn't make sense. Cheers. $\endgroup$
    – Steve
    Mar 20 '21 at 11:19
  • 1
    $\begingroup$ Erp there's a typo in there.. you'll need: path("${sample_name}.log") into test1_logs if your log's filename fits that format. Whether it does or not, you'll need to check. HTH. $\endgroup$
    – Steve
    Mar 20 '21 at 14:18
  • 1
    $\begingroup$ What's the name of your log file? I haven't run plink before, so I just used "${sample_name}.log" as an example. If the name of the log file is 'plink.log' then you'd use: path("plink.log") into test1_logs in your output declaration. $\endgroup$
    – Steve
    Mar 21 '21 at 3:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.