Supposing upcoming RNA-seq for 3 conditions and 3 replications for each, how much is data storage requirements?
$\begingroup$
$\endgroup$
6
-
3$\begingroup$ What sort of sequencing depth and read length? $\endgroup$– Devon RyanJan 23, 2019 at 10:54
-
$\begingroup$ Thank you let's say 75pb, paired end and 30X $\endgroup$– ExhaustedJan 23, 2019 at 11:02
-
$\begingroup$ I assume you mean 30 million reads (pairs really), since 30X is meaningless for RNA-seq. $\endgroup$– Devon RyanJan 23, 2019 at 11:03
-
$\begingroup$ Sorry I am very bad in these staffs :( thank you for correcting me $\endgroup$– ExhaustedJan 23, 2019 at 11:11
-
2$\begingroup$ Please add all the extra information to your question since comments are easy to miss and can be deleted without warning. Also please clarify what species you will be working on since that can radically affect how much space you will need. $\endgroup$– terdon ♦Jan 23, 2019 at 13:55
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
5
The breakdown would be:
- 1.5-2 GB per sample for fastq files.
- ~3GB per BAM file, depending on the aligner
So ~5GB per BAM file, which is about 50GB total. That can vary by maybe 20%, depending on how compressible the results are (I rearrange our fastq files to make them more compressible, so I'm probably underestimating what others would see).
Obviously using Salmon/Kallisto would halve the storage requirements. Using CRAM would also decrease things, though obviously fewer tools support it.
answered Jan 23, 2019 at 11:09
-
$\begingroup$ So, for a whole genome sequencing that would roughly 15 TB? $\endgroup$ Jan 23, 2019 at 11:12
-
1$\begingroup$ You would need 600-750 million pairs for a human genome (mostly depending on the fragment size), which is 20-25 times as much data. $\endgroup$ Jan 23, 2019 at 11:21
-
$\begingroup$ Maybe it is for a different question but how do you rearrange the fastq to make them more compressible? Is there some paper or blog to read on this? $\endgroup$– llrsJan 23, 2019 at 14:50
-
1$\begingroup$ @llrs It's a by-product of using
clumpify(from BBTools) to remove optical duplicates. The process involves sorting fastq files such that similar reads are near each other, which means the resulting files compress more than they would otherwise. I run that as part of our normal Illumina demultiplexing pipeline. $\endgroup$ Jan 23, 2019 at 15:04 -
$\begingroup$ Ok, then it might be out of my scope (I usually get the demultiplexed files). Thanks for the tip! $\endgroup$– llrsJan 24, 2019 at 8:18