# SyntaxError after end of Snakefile code?

I am trying to run Fastqc on multiple files using Snakemake but am receiving an ambiguous SyntaxError past the last line of code in the Snakefile.

Snakefile:

import pandas as pd

SAMPLES=SAMPLE_TABLE["sample_name"]
SAMPLES=list(SAMPLES)
DIRECTORY=list(SAMPLE_TABLE["FolderName"])

rule run_fastqc:
input:
expand("/PathToInputsFolder/{dir}/{sample}_{{Rnumber}}_001.fastq.gz", zip, dir=DIRECTORY, sample=SAMPLES)
output:
expand("{sample}_{{Rnumber}}_001.fastqc.html", sample=SAMPLES)
expand("{sample}_{{Rnumber}}_001.fastqc.zip", sample=SAMPLES)
shell:
"fastqc {input} --outdir=./qc/"


Code:

snakemake --cores 1 {.+}.fastqc.html {.+}.fastqc.zip


Output:

SyntaxError in line 16 of /PathToSnakefile/Snakefile:
invalid syntax (Snakefile, line 16)


How should I figure out where the error is? Thank you for your time!

You're missing a comma between the two output lines. (John Marshall's more detailed syntax error message was spot-on.)

With a simplified version of your Snakefile:

SAMPLES=["s1", "s2", "s3"]
rule run_fastqc:
output:
expand("{sample}_{{Rnumber}}_001.fastqc.html", sample=SAMPLES)
expand("{sample}_{{Rnumber}}_001.fastqc.zip", sample=SAMPLES)
shell:
"echo {output}"


You get the syntax error:

$snakemake -nrp s1_R1_001.fastqc.html SyntaxError in line 6 of /data/home/jesse/Snakefile: invalid syntax (Snakefile, line 6)  With the comma: SAMPLES=["s1", "s2", "s3"] rule run_fastqc: output: expand("{sample}_{{Rnumber}}_001.fastqc.html", sample=SAMPLES), # <-- here expand("{sample}_{{Rnumber}}_001.fastqc.zip", sample=SAMPLES) shell: "echo {output}"  Now it's OK: $ snakemake -nrp s1_R1_001.fastqc.html
Building DAG of jobs...
Job stats:
----------  -------  -------------  -------------
run_fastqc        1              1              1
total             1              1              1

[Fri Nov 12 08:54:54 2021]
rule run_fastqc:
output: s1_R1_001.fastqc.html, s2_R1_001.fastqc.html, s3_R1_001.fastqc.html, s1_R1_001.fastqc.zip, s2_R1_001.fastqc.zip, s3_R1_001.fastqc.zip
jobid: 0
reason: Missing output files: s1_R1_001.fastqc.html
wildcards: Rnumber=R1
resources: tmpdir=/tmp

echo s1_R1_001.fastqc.html s2_R1_001.fastqc.html s3_R1_001.fastqc.html s1_R1_001.fastqc.zip s2_R1_001.fastqc.zip s3_R1_001.fastqc.zip
Job stats:
----------  -------  -------------  -------------
run_fastqc        1              1              1
total             1              1              1

This was a dry-run (flag -n). The order of jobs does not reflect the order of execution.


Each expand() makes a list, so at first Snakemake sees two lists back-to-back and doesn't know what to do with them. If you use a comma and give output names you could handle the two lists separately in your rule, but if you don't need to, you could also just us a single expand call, like:

output:
expand("{sample}_{{Rnumber}}_001.fastqc.{ext}", sample=SAMPLES, ext=["html", "zip"])


Other suggestions for troubleshooting:

• If the above simplified Snakefile works but the real one still doesn't, edit them incrementally to be more and more similar and you'll hone in on whatever problems are left.
• I find trimming away complexity until the problem reveals itself is often easiest, so I'd keep simplifying the real case until that happens.
• expand is just a Python function that Snakemake provides you, so you can call it yourself outside of a rule definition and check what it returns.
• You can even comment out all the Snakemake-specific syntax and leave just Python code when debugging, to sanity-check the rest of the code (e.g. your inputs and outputs from expand).
• You can give someone a fish [a comma] today… or you can teach them to fish [to decrypt their own error messages]… Nov 12 '21 at 14:31
• With the suggested comma, I get the message: "WorkflowError: Duplicate output file pattern in rule run_fastqc. First two duplicate for entries 0 and 1 File "/PathToSnakefile/Snakefile", line 22, in <module>". I'm not sure how the output files are duplicate if one is .html and the other is .zip. I also added 'READ=list(SAMPLE_TABLE["read"])' and changed '{{Rnumber}}' to 'R{val}, val=READ' so the wildcard is defined, and it still leads to the duplicate output error :/ Nov 12 '21 at 23:13
• I added a few bullets on the end of my answer that might help you. To John Marshall's point, Snakemake is correctly telling you the problem here, but you're assuming the output files are what you expect them to be. If you investigate the expand behavior in your new version you'll see the issue. Nov 16 '21 at 19:18

How to figure out what error messages are trying to tell you is one of the great mysteries… :-)

What version of Snakemake are you using? I get slightly different messages. With an ancient version (3.13.3) on one machine:

SyntaxError in line 14 of /path/to/Snakefile:
invalid syntax


With a more up-to-date version (6.9.1, running on Python 3.9):

SyntaxError in line 16 of /path/to/Snakefile:
invalid syntax. Perhaps you forgot a comma? (Snakefile, line 16)


Both of these are useful hints. So one piece of advice would be to update your Snakemake and/or Python if it is old.

Another useful technique when faced with a mysterious error like this is to temporarily remove parts of your Snakefile until the error goes away. The complicated part of your script towards the end of the file is the output: stanza, so you might try removing one of the expand(…) lines and seeing what happens. If the error goes away, then you have localised the problem and can put the line back and carefully check the syntax in that particular part of the Snakefile.