12
votes
What's the best way to download data from the SRA? Is it really this slow?
Proximity to NCBI may not necessarily give you the fastest transfer speed. AWS may be deliberately throttling the Internet connection to limit the likelihood that people will use it for undesirable ...
- 13.8k
9
votes
Accepted
How to validate that BAMs have been downloaded correctly?
samtools quickcheck is all you need. From the manual:
Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header ...
- 19.6k
9
votes
Accepted
How to download the whole BLAST nt database into a specific folder?
-O is to specify the name of the output file. You want the -P option.
So, from your example
...
- 1,816
7
votes
How do you query and explore ENCODE data?
I don't know whether there is an API, but ENCODE's website does provide an interactive data matrix where you can filter data based on assay and sample type, place data sets in a "shopping cart", and ...
- 5,060
6
votes
What's the best way to download data from the SRA? Is it really this slow?
In general, the best way to download SRA data is: don't download from SRA. However, as ENA has not be sync'd yet, I would recommend to download from SRA ftp and then convert to fastq locally. You can ...
- 6,485
5
votes
Accepted
Convert SRA to FastA
Finally, I found an alternative to the SRA translation: a link that works! For those of you interested in knowing how to download FastA files from NCBI using an accession number, try the following ...
- 221
5
votes
Accepted
How to download the whole directory of an ensembl FTP page?
You can use the mirroring function of wget:
wget -m ftp://ftp.ensembl.org/pub/release-91/embl/bos_taurus/
- 218
5
votes
Downloading URL using AWK for fixed fields
awk '{print "wget " $9 " -O " $1 "_" $2 ".fastq.gz"}' tmp
will give you
...
- 2,574
5
votes
How do you query and explore ENCODE data?
You can just add /?format=json to any page to get the JSON output.
ENCODE REST API documentation: https://www.encodeproject.org/help/rest-api/
Example scripts: ...
- 2,169
4
votes
Stable download URLs
The Persistent uniform resource locator or PURL is one such solution, these are designed to be a bit more robust than permalinks in so much as they are supposed to survive the change of domain name. ...
- 1,059
4
votes
Accepted
Download multiple SRA files
A quick look at your link tells me the SRR numbers run from SRR837819 to SRR837856. You can use fastq-dump from the sratoolkit, and make a for loop around it in bash.
Something like this should work:
...
- 3,571
4
votes
Downloading URL using AWK for fixed fields
A variation of Bioathlete’s answer that doesn't suffer from escaping issues:
...
- 141
4
votes
Download multiple fastq files using fastq-dump
A sample code is given in the salmon documentation as follows. Source
...
- 76
4
votes
Accepted
Replacing column 1 in file1 with matching ID's in file 2
I was not sure about the exact output (columns) you desired. The following code should inner merge both data frames. It writes a copy of 'model_list' column as a first column. You could remove any ...
- 596
3
votes
How to download gene expression data from NCBI gene database
You can download raw data from SRA (NCBI). On this website under RNA-Seq alignments, you'll find the samples. Click on them, for example SAMN05231885, and then click on PRJNA325427 to see the link to ...
- 3,571
3
votes
Download multiple SRA files
I suggest you follow the advice in Eric A Brenner's answer and just download the fastq files. However, if you really really want to use the SRA files for some reason, note that you can use parallel-...
- 19.6k
3
votes
Download multiple SRA files
Assuming you ultimately just want the fastq files and you know the SRR (run) numbers, I would download them from here: ftp://ftp.sra.ebi.ac.uk/vol1/fastq/
As for downloading multiple files, I've just ...
3
votes
What's the best way to download data from the SRA? Is it really this slow?
By far the fastest method in my experience has been to use the SRAdb library in R. For most entries, you can download fastq files directly. Some older experiments don't have them, but I've still found ...
- 372
3
votes
Accepted
How to dump genes from GenBank in GFF3 format?
Turns out you can just grab the GFF3 from the NCBI's FTP site!
ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/003/195/GCA_000003195.3_Sorghum_bicolor_NCBIv3/
See:
https://www.ncbi.nlm.nih.gov/...
- 612
3
votes
Accepted
Download or create VCF from Human Genome Diversity Project (HGDP)
I presume you are looking for the recently sequenced WGS data rather than the genotype data? If so, the different files can be found here: ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/
...
- 1,679
3
votes
What happened to sra ftp server?
SRA moved all of the data to the cloud per https://ncbiinsights.ncbi.nlm.nih.gov/2020/02/24/sra-cloud/
You can still download data for free using sratoolkit as ...
- 1,266
3
votes
Accepted
How to wget a gff from NCBI database?
my suggestion would be to go via the assembly page.
For your example link, I would
Click the link to the BioProject: PRJNA164
On the right side, under "Related information" choose the ...
3
votes
Accepted
What is the NCBI's definition of an "atypical genome"?
Biostars user MirianT_NCBI provided an answer on my crosspost, which I'm copying here:
You can find more information about what constitute an atypical genome in this FAQ. Here's the list of reasons ...
- 613
3
votes
Where to download a file with major and minor alleles at every position?
Sounds like you're looking for the dbSNp files. You haven't specified a species, but you can download the data for human from the dbSNP FTP site. Specifically, you want:
...
- 9,662
2
votes
What's the best way to download data from the SRA? Is it really this slow?
NCBI is moving slowly to cloud (AWS/Google) for hosting. As such the tradiational FTP urls do not always work. My solution involves using pysradb.
An entire project can be downloaded by:
...
- 991
2
votes
SNP data Download Help
I would recommend to do this in R, using biomaRt.
For example if you want all the SNPs from IDH1 (ENSG00000138413) gene:
...
- 3,571
2
votes
How to download SNP data from specific regions, population and positions?
The HapMap website was suspended due to a security issue. Archived HapMap data is available via FTP, but it's a better idea to download up-to-date International Genome data from the IGSR website (...
- 13.8k
2
votes
Accepted
How to link GDC ids to CCLE cell line names?
The filenames of those bams contain the cell line names. They are constructed:
SAMPLEID.CELL_LINE_NAME.REPLICATE.bam
Thus the first 5 cell lines in your screen capture are SK-N-F1, TE-9, CJM, HCI-...
- 3,301
2
votes
Convert SRA to FastA
Here is a Biopython solution via Entrez utilies, simply copy the code into a python script and execute it:
...
- 3,928
2
votes
How do you query and explore ENCODE data?
So let's say I want to get the bed file of this experiment using curl encodeproject.org/experiments/ENCSR000CKC
You can also search for files that belong to a ...
- 21
Only top scored, non community-wiki answers of a minimum length are eligible