3
$\begingroup$

I am trying to download multiple SRA files located in several SRA accessions. Some of my accession numbers are as follows:

SRR1002678
SRR1158898
SRR1158907
SRR1158950
SRR1181152
SRR1181155
SRR1181196
SRR1181204
SRR1181205
SRR1181208
SRR1269497
SRR2100227
SRR2100419
SRR2100830
SRR2101344
SRR671725
SRR671726
SRR671740
SRR671759
SRR671760
SRR671761
SRR671779
SRR671782
SRR671815
SRR671821
SRR671842
SRR671862
SRR671863
SRR671864
SRR671865
.............

I tried the following command:

for i in {SRR1002678..SRR1184123}; do wget ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/SRR1*/$i; done

but this did not work.

I tried prefetch as well:

sort -u SRR.txt | parallel -j 4 "prefetch {}"

where SRR.txt is the list of my SRA IDs. However, with this command only a few files are downloaded as *.sra files, while with several other SRA IDs I get the following error:

2018-11-15T06:17:20 prefetch.2.8.2: 1) Downloading 'SRR1183962'...
2018-11-15T06:17:20 prefetch.2.8.2:  Downloading via https...
2018-11-15T06:20:21 prefetch.2.8.2: 1) 'SRR1183962' was downloaded successfully
2018-11-15T06:20:27 prefetch.2.8.2: 'SRR1183962' has 0 unresolved dependencies
2018-11-15T06:20:38 prefetch.2.8.2: 'SRR1183962' has no remote vdbcache

2018-11-15T06:18:59 prefetch.2.8.2: 1) Downloading 'SRR1184034'...
2018-11-15T06:18:59 prefetch.2.8.2:  Downloading via https...
2018-11-15T06:20:40 prefetch.2.8.2: 1) failed to download SRR1184034
2018-11-15T06:20:40 prefetch.2.8.2 sys: libs/kns/unix/syssock.c:606:KSocketTimedRead: timeout exhausted while reading file within network system module - mbedtls_ssl_read returned -76 ( NET - Reading information from the socket failed )
2018-11-15T06:20:40 prefetch.2.8.2 int: libs/kns/unix/syssock.c:606:KSocketTimedRead: timeout exhausted while reading file within network system module -

I do not have a range of SRA files to download but quite random SRA files to download. What am I doing wrong? Any help is highly appreciated.

$\endgroup$
6
$\begingroup$

First of all, the brace expansion you tried to use isn't valid. While bash can expand {1..3} to 1 2 3, it has no way of knowing how to expand {SRR1002678..SRR1184123}:

$ echo {1..3}
1 2 3
$ echo {SRR1002678..SRR1184123}
{SRR1002678..SRR1184123}

Perhaps you meant SRR{1002678..1184123}? But there's no need for this, you already have the accessions in a file and can read that. I see that the FTP server you are accessing has organized the data like this:

sra/sra-instant/reads/ByRun/sra/SRR/SRRNNN/SRRNNNXXXX/SRRNNNXXXX.sra

Where NNN are the 1st three digits of the accession. So, for example, the file for accession SRR1002678 is at:

ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/SRR118/SRR1181152/SRR1181152.sra

With this in mind, you can write a little script that extracts the URL from each accession:

while read i; do 
    n=$(echo "$i" | sed -E 's/(SRR...).*/\1/'); 
    wget ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/$n/$i/$i.sra
done < SRR.txt

The only trick is the sed command which simply looks for SRR and the three next characters and removes everything else on the line (it "captures" the found pattern and then replaces the line with what was captured (\1)).

$\endgroup$
  • $\begingroup$ Thanks @terdon but im getting the same kind of error please see: SRR1158898%0D/SRR1158898%0D.sra => ‘SRR1158898%0D.sra’ Resolving ftp.ncbi.nlm.nih.gov (ftp.ncbi.nlm.nih.gov)... 130.14.250.11, 2607:f220:41e:250::13 Connecting to ftp.ncbi.nlm.nih.gov (ftp.ncbi.nlm.nih.gov)|130.14.250.11|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /sra/sra-instant/reads/ByRun/sra/SRR/SRR115/SRR1158898\r ... No such directory ‘sra/sra-instant/reads/ByRun/sra/SRR/SRR115/SRR1158898\r’. $\endgroup$ – Safina A.R Nov 16 '18 at 9:39
  • 2
    $\begingroup$ @SafinaA.R looks like your file was created in Windows, not Linux. Note the \r at the end of the path in the error message. Fix the line endings by running sed -i 's/\r//' SRR.txt and try again. $\endgroup$ – terdon Nov 16 '18 at 10:23
  • 1
    $\begingroup$ @SafinaA.R please don't do this in the comments. They are very hard to read and shouldn't be used for discussion. Either edit your question and show the commands you tried and how they failed or come into our chat room. $\endgroup$ – terdon Nov 16 '18 at 11:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.