Snakemake scatter-gather with wildcard AmbiguousRuleException

Question

My problem is when using Snakemake scatter-gather feature the documentation is basic and i modified my code according to mentioned in this link

rule fastq_fasta:
    input:rules.trimmomatic.output.out_file
    output:"data/trimmed/{sample}.fasta"
    shell:"sed -n '1~4s/^@/>/p;2~4p' {input} > {output}"

rule split:
    input:
        "data/trimmed/{sample}.fasta"
    params:
        scatter_count=config["scatter_count"],
        scatter_item = lambda wildcards: wildcards.scatteritem
    output:
        temp(scatter.split("data/trimmed/{{sample}}_{scatteritem}.fasta"))
    script:
        "scripts/split_files.py"
        
rule process:
    input:"data/trimmed/{sample}_{scatteritem}.fasta"
    output:"data/processed/{sample}_{scatteritem}.csv"
    script:
        "scripts/process.py"

rule gather:
    input:
        gather.split("data/processed/{{sample}}_{scatteritem}.csv")
    output:
        "data/processed/{sample}.csv"
    shell:
        "cat {input} > {output}"

I added wildcard option but, i got AmbiguousRuleException: Rules fastq_to_fasta(which is previous rule) and split are ambiguous for the file data/trimmed/Ornek_411-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-8.fasta

I tried lots of things but either rules are not calling or take AmbiguousRuleException. What am i missing, can someone help?

Answer 1

It's not a problem with the scatter-gather feature per-se; Snakemake is just confusing itself with what text should match what wildcards with these filenames. You could fix this by defining a wildcard constraint for your samples, and/or changing your filename patterns to clarify what should match what. The docs on scatter-gather show an example with different directory names and file suffixes that prevent this from happening. Probably the simplest fix would be to follow that idea and use different directory names for the split parts, like trimmed-split and processed-split.

A hint of what's happening is the repeating ...-of-81-... thing, and the mention of ambiguity between rules. (See also the --debug-dag option.) If you have a sample named Ornek_411 and you want to make data/processed/Ornek_411.csv, Snakemake will look for data/processed/Ornek_411_1-of-N.csv and so on, and try to figure out how to make those. But a filename like that could be the output of either the process rule (sample=Ornek_411 and scatteritem=1-of-N) or the gather rule itself again (sample=Ornek_411_1-of-N). Obviously you want the former and not the latter, but it doesn't know that automatically.