10
$\begingroup$

I would like to do easily reproducible analysis using publicly available data from NCBI, so I have chosen a snakemake.

I would like to write a single rule, that would be able to download any genome given a species code name and separated table of species and their NCBI IDs. So I wrote a script scripts/download.sh that takes a species code and download the genome to data/<species_code>/genome.fa.gz. The script internally reads the table tables/download_table.tsv, where are corresponding species code names and NCBI IDs.

So I tried to do a snakemake like this :

species='Cbir Avag Fcan Lcla Dcor Dpac Pdav Psp62 Psp79 Minc1 Minc2 Mjav Mare Mflo Mhap Pant'

rule download:
    input :
        "tables/download_table.tsv"
    output :
        "data/{sp}/genome.fa.gz"
    shell :
        "scripts/download.sh {sp}"

However, snakemake returned an error message I do not really understand :

Target rules may not contain wildcards. Please specify concrete files or a rule without wildcards.`.

Is there a way to write a single rule for downloading all the genomes?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
9
$\begingroup$

The problem is that you need a master rule that requires all of your desired outputs as inputs, in your case it would be :

rule all:
    input:
        expand("data/{sp}/genome.fa.gz", sp=species.split(' '))

You'll also need separate download link inputs for each species. You could make a separate download_table.tsv for each species, but it would probably be easier to make a config file with this information, and add a params keyword to your rule. Something like:

rule download:
   params:
       url=config['locations']['sp']
   output :
       "data/{sp}/genome.fa.gz"
   shell :
       "scripts/download.sh {params.url}"
Improve this answer
$\endgroup$
5
  • 3
    $\begingroup$ I wouldn't erase it if I were you. A lot of people get confused about Snakemake error messages, so its helpful to have some examples on here. $\endgroup$
    – heathobrien
    Nov 3, 2017 at 15:45
  • 1
    $\begingroup$ Then I will at least dramatically edit it. (I try to do it in a way so your answer still fits) $\endgroup$
    – Kamil S Jaron
    Nov 3, 2017 at 15:47
  • $\begingroup$ @KamilSJaron I agree the question should be kept, but maybe change the title to reflect the error message, this has nothing to do with NCBI $\endgroup$
    – Chris_Rands
    Nov 3, 2017 at 19:13
  • $\begingroup$ Also note that Snakemake has an NCBI remote provider: snakemake.readthedocs.io/en/stable/snakefiles/… $\endgroup$
    – Johannes Köster
    Nov 6, 2017 at 10:51
  • $\begingroup$ @JohannesKöster I was looking at it. But since I could not make it work without remote I have not even tried with remote. And then the problem I originally had with non-remote solution is very much unrelated to downloading of sequences. It's a good tip though, maybe work an answer? $\endgroup$
    – Kamil S Jaron
    Nov 10, 2017 at 15:53
3
$\begingroup$

Given the Makefile you provided and which someone then deleted from the question, you should probably just add this line (this should be the first rule):

rule all: input: ["data/{}/genome.fa.gz".format(x) for x in species.split()]

This rule just specifies a list of expected output files and corresponds to the following lines from the original Makefile:

GENOMES=$(patsubst %, data/%/genome.fa.gz, $(SPECIES)) all : $(GENOMES)

Improve this answer
$\endgroup$
1
  • $\begingroup$ I have deleted Make equivalent since was not really related to the problem. $\endgroup$
    – Kamil S Jaron
    Nov 5, 2017 at 11:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.