2
$\begingroup$

My problem is when using Snakemake scatter-gather feature the documentation is basic and i modified my code according to mentioned in this link

rule fastq_fasta:
    input:rules.trimmomatic.output.out_file
    output:"data/trimmed/{sample}.fasta"
    shell:"sed -n '1~4s/^@/>/p;2~4p' {input} > {output}"

rule split:
    input:
        "data/trimmed/{sample}.fasta"
    params:
        scatter_count=config["scatter_count"],
        scatter_item = lambda wildcards: wildcards.scatteritem
    output:
        temp(scatter.split("data/trimmed/{{sample}}_{scatteritem}.fasta"))
    script:
        "scripts/split_files.py"
        
rule process:
    input:"data/trimmed/{sample}_{scatteritem}.fasta"
    output:"data/processed/{sample}_{scatteritem}.csv"
    script:
        "scripts/process.py"

rule gather:
    input:
        gather.split("data/processed/{{sample}}_{scatteritem}.csv")
    output:
        "data/processed/{sample}.csv"
    shell:
        "cat {input} > {output}"

I added wildcard option but, i got AmbiguousRuleException: Rules fastq_to_fasta(which is previous rule) and split are ambiguous for the file data/trimmed/Ornek_411-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-8.fasta

I tried lots of things but either rules are not calling or take AmbiguousRuleException. What am i missing, can someone help?

$\endgroup$

1 Answer 1

1
$\begingroup$

It's not a problem with the scatter-gather feature per-se; Snakemake is just confusing itself with what text should match what wildcards with these filenames. You could fix this by defining a wildcard constraint for your samples, and/or changing your filename patterns to clarify what should match what. The docs on scatter-gather show an example with different directory names and file suffixes that prevent this from happening. Probably the simplest fix would be to follow that idea and use different directory names for the split parts, like trimmed-split and processed-split.

A hint of what's happening is the repeating ...-of-81-... thing, and the mention of ambiguity between rules. (See also the --debug-dag option.) If you have a sample named Ornek_411 and you want to make data/processed/Ornek_411.csv, Snakemake will look for data/processed/Ornek_411_1-of-N.csv and so on, and try to figure out how to make those. But a filename like that could be the output of either the process rule (sample=Ornek_411 and scatteritem=1-of-N) or the gather rule itself again (sample=Ornek_411_1-of-N). Obviously you want the former and not the latter, but it doesn't know that automatically.

$\endgroup$
1
  • $\begingroup$ Firstly thanks changing directory name pattern did trick for me. Thanks for explanation of debug-dag option too. i am aware of this logic because i got that before but with just wildcard but i couldn't set scatteritem logic $\endgroup$
    – sahin
    Jul 21, 2022 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.