Skip to main content
12 votes

What's the best way to download data from the SRA? Is it really this slow?

Proximity to NCBI may not necessarily give you the fastest transfer speed. AWS may be deliberately throttling the Internet connection to limit the likelihood that people will use it for undesirable ...
gringer's user avatar
  • 14.3k
9 votes
Accepted

How to validate that BAMs have been downloaded correctly?

samtools quickcheck is all you need. From the manual: Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header ...
Devon Ryan's user avatar
  • 19.6k
9 votes
Accepted

How to download the whole BLAST nt database into a specific folder?

-O is to specify the name of the output file. You want the -P option. So, from your example ...
heathobrien's user avatar
  • 1,816
7 votes

How do you query and explore ENCODE data?

I don't know whether there is an API, but ENCODE's website does provide an interactive data matrix where you can filter data based on assay and sample type, place data sets in a "shopping cart", and ...
Daniel Standage's user avatar
6 votes

What's the best way to download data from the SRA? Is it really this slow?

In general, the best way to download SRA data is: don't download from SRA. However, as ENA has not be sync'd yet, I would recommend to download from SRA ftp and then convert to fastq locally. You can ...
user172818's user avatar
  • 6,535
5 votes
Accepted

Convert SRA to FastA

Finally, I found an alternative to the SRA translation: a link that works! For those of you interested in knowing how to download FastA files from NCBI using an accession number, try the following ...
Poshi's user avatar
  • 221
5 votes
Accepted

How to download the whole directory of an ensembl FTP page?

You can use the mirroring function of wget: wget -m ftp://ftp.ensembl.org/pub/release-91/embl/bos_taurus/
Michael Kuhn's user avatar
5 votes

Downloading URL using AWK for fixed fields

awk '{print "wget " $9 " -O " $1 "_" $2 ".fastq.gz"}' tmp will give you ...
Bioathlete's user avatar
  • 2,574
5 votes

How do you query and explore ENCODE data?

You can just add /?format=json to any page to get the JSON output. ENCODE REST API documentation: https://www.encodeproject.org/help/rest-api/ Example scripts: ...
burger's user avatar
  • 2,199
4 votes

Stable download URLs

The Persistent uniform resource locator or PURL is one such solution, these are designed to be a bit more robust than permalinks in so much as they are supposed to survive the change of domain name. ...
Matt Bashton's user avatar
  • 1,069
4 votes
Accepted

Download multiple SRA files

A quick look at your link tells me the SRR numbers run from SRR837819 to SRR837856. You can use fastq-dump from the sratoolkit, and make a for loop around it in bash. Something like this should work: ...
benn's user avatar
  • 3,571
4 votes

Downloading URL using AWK for fixed fields

A variation of Bioathlete’s answer that doesn't suffer from escaping issues: ...
David Foerster's user avatar
4 votes

Download multiple fastq files using fastq-dump

A sample code is given in the salmon documentation as follows. Source ...
Balan's user avatar
  • 76
4 votes
Accepted

Replacing column 1 in file1 with matching ID's in file 2

I was not sure about the exact output (columns) you desired. The following code should inner merge both data frames. It writes a copy of 'model_list' column as a first column. You could remove any ...
Supertech's user avatar
  • 606
3 votes

How to download gene expression data from NCBI gene database

You can download raw data from SRA (NCBI). On this website under RNA-Seq alignments, you'll find the samples. Click on them, for example SAMN05231885, and then click on PRJNA325427 to see the link to ...
benn's user avatar
  • 3,571
3 votes

Download multiple SRA files

I suggest you follow the advice in Eric A Brenner's answer and just download the fastq files. However, if you really really want to use the SRA files for some reason, note that you can use parallel-...
Devon Ryan's user avatar
  • 19.6k
3 votes

Download multiple SRA files

Assuming you ultimately just want the fastq files and you know the SRR (run) numbers, I would download them from here: ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ As for downloading multiple files, I've just ...
Eric A Brenner's user avatar
3 votes

What's the best way to download data from the SRA? Is it really this slow?

By far the fastest method in my experience has been to use the SRAdb library in R. For most entries, you can download fastq files directly. Some older experiments don't have them, but I've still found ...
Sarah Carl's user avatar
3 votes
Accepted

How to dump genes from GenBank in GFF3 format?

Turns out you can just grab the GFF3 from the NCBI's FTP site! ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/003/195/GCA_000003195.3_Sorghum_bicolor_NCBIv3/ See: https://www.ncbi.nlm.nih.gov/...
Dan's user avatar
  • 612
3 votes
Accepted

Download or create VCF from Human Genome Diversity Project (HGDP)

I presume you are looking for the recently sequenced WGS data rather than the genotype data? If so, the different files can be found here: ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/ ...
user438383's user avatar
  • 1,689
3 votes

What happened to sra ftp server?

SRA moved all of the data to the cloud per https://ncbiinsights.ncbi.nlm.nih.gov/2020/02/24/sra-cloud/ You can still download data for free using sratoolkit as ...
vkkodali's user avatar
  • 1,266
3 votes
Accepted

How to wget a gff from NCBI database?

my suggestion would be to go via the assembly page. For your example link, I would Click the link to the BioProject: PRJNA164 On the right side, under "Related information" choose the ...
Adrian Sven Geissler's user avatar
3 votes
Accepted

What is the NCBI's definition of an "atypical genome"?

Biostars user MirianT_NCBI provided an answer on my crosspost, which I'm copying here: You can find more information about what constitute an atypical genome in this FAQ. Here's the list of reasons ...
acvill's user avatar
  • 613
3 votes

Where to download a file with major and minor alleles at every position?

Sounds like you're looking for the dbSNp files. You haven't specified a species, but you can download the data for human from the dbSNP FTP site. Specifically, you want: ...
terdon's user avatar
  • 10.2k
2 votes

What's the best way to download data from the SRA? Is it really this slow?

NCBI is moving slowly to cloud (AWS/Google) for hosting. As such the tradiational FTP urls do not always work. My solution involves using pysradb. An entire project can be downloaded by: ...
Saket Choudhary's user avatar
2 votes

SNP data Download Help

I would recommend to do this in R, using biomaRt. For example if you want all the SNPs from IDH1 (ENSG00000138413) gene: ...
benn's user avatar
  • 3,571
2 votes

How to download SNP data from specific regions, population and positions?

The HapMap website was suspended due to a security issue. Archived HapMap data is available via FTP, but it's a better idea to download up-to-date International Genome data from the IGSR website (...
gringer's user avatar
  • 14.3k
2 votes
Accepted

How to link GDC ids to CCLE cell line names?

The filenames of those bams contain the cell line names. They are constructed: SAMPLEID.CELL_LINE_NAME.REPLICATE.bam Thus the first 5 cell lines in your screen capture are SK-N-F1, TE-9, CJM, HCI-...
Ian Sudbery's user avatar
  • 3,321
2 votes

Convert SRA to FastA

Here is a Biopython solution via Entrez utilies, simply copy the code into a python script and execute it: ...
Chris_Rands's user avatar
  • 3,948
2 votes

How do you query and explore ENCODE data?

So let's say I want to get the bed file of this experiment using curl encodeproject.org/experiments/ENCSR000CKC You can also search for files that belong to a ...
ksgraham's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible