5
$\begingroup$

I am trying to index wheat genome using STAR through following command

STAR --runMode genomeGenerate --genomeFastaFiles Triticum_aestivum.TGACv1.dna_sm.toplevel.fa --runThreadN 28

But getting following error,

terminate called after throwing an instance of 'std::bad_alloc'
what():  std::bad_alloc
Aborted (core dumped)

The size of the genome is 13.5 GB and the server capacity is 32 cores with 125 GB RAM, how this error can be resolved?

Memory usage: enter image description here

I have decreased the number of threads to 8 but again getting the same error as shown below:

> STAR   --runMode genomeGenerate   --runThreadN 8   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                      
Jun 04 02:13:31 ..... started STAR run                                                                                                                                                                                           
Jun 04 02:13:31 ... starting to generate Genome files                                                                                                                                                                            
terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  
what():  std::bad_alloc                                                                                                                                                                                                        
Aborted (core dumped)

Even I have tried to use STAR without any threads but still the same error:

> STAR   --runMode genomeGenerate   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                         
Jun 04 02:19:44 ..... started STAR run                                                                                                                                                                                           
Jun 04 02:19:44 ... starting to generate Genome files                                                                                                                                                                            
terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  
what():  std::bad_alloc
Aborted (core dumped)

May there can be an issue in reference of wheat that I am using, I got this reference from here. Here are the first ten lines of the reference using head command:

> head wheat.fa
>Pt dna:chromosome chromosome:TGACv1:Pt:1:135835:1 REF                                                                                                                                                                           
ACAGAAATACCCAATATCTTGTTCTAGCAAGATATTGGGTATTTCTGTCTTTTCTTTCTT                                                                                                                                                                     
CAAAAATTCTTATATGTTAGCAGAAAAACCTTATCCATTAATAGATGGAACTTCAACAGC                                                                                                                                                                     
AGCTAAGTCTAGAGGGAAGTTGTGAGCATTACGTTCGTGCATTACTTCCATACCAAGGTT                                                                                                                                                                     
AGCACGGTTGATGATATCAGCCCAAGTATTAATAACGCGACCTTGACTATCAACTACAGA                                                                                                                                                                     
TTGGTTGAAATTGAAACCATTTAGGTTGAAAGCCATAGTACTAATACCTAAAGCAGTGAA                                                                                                                                                                     
CCAGATTCCTACTACAGGCCAAGCAGCCAAGAAGAAGTGTAAAGAACGAGAGTTGTTGAA                                                                                                                                                                     
ACTAGCATATTGGAAGATTAATCGGCCAAAATAACCATGAGCAGCCACAATATTATAAGT                                                                                                                                                                     
TTCTTCCTCTTGACCAAATTTGTAACCCTCATTAGCAGATTCATTTTCAGTAGTTTCCCT                                                                                                                                                                     
GATCAAACTAGAGGTTACCAAGGAACCATGCATAGCACTGAATAGGGAACCGCCGAATAC

How this problem can be resolved?

Improve this question
$\endgroup$
7
  • $\begingroup$ Keep an eye on the memory usage and see if you're actually exceeding what's available. $\endgroup$
    – Devon Ryan
    Commented May 23, 2018 at 9:48
  • 1
    $\begingroup$ The memory does not exceeds 1.6 usage that is normal, $\endgroup$
    – Ammar Sabir Cheema
    Commented May 23, 2018 at 9:58
  • 1
    $\begingroup$ Here is the link to the memory usage $\endgroup$
    – Ammar Sabir Cheema
    Commented May 23, 2018 at 10:14
  • $\begingroup$ Try removing the --sjdbGTF... stuff from your command as it's likely there's an issue either with the GFF3 file or the fasta file. $\endgroup$
    – Devon Ryan
    Commented May 23, 2018 at 11:51
  • 1
    $\begingroup$ Use (much!) fewer threads. Threads speed up processing, but additional threads increase the memory requirement linearly. $\endgroup$
    – Konrad Rudolph
    Commented May 23, 2018 at 13:16

2 Answers 2

Reset to default
3
$\begingroup$

In addition to the other answer (to reduce the number of threads), you might also need to add a --genomeChrBinNbits argument. Your reference has more than 5000 scaffolds, so you need to adjust the --genomeChrBinNbits argument to reduce RAM consumption. Found here.

They recommend:

--genomeChrBinNbits} = min(18, log2(GenomeLength/NumberOfReferences))

In your case:

Genome size 17G, number of scaffolds 735945

So --genomeChrBinNbits should be set to log2(17,000,000,000/735,945) = 14.4955, so let's say 14 or 15.

Improve this answer
$\endgroup$
2
  • $\begingroup$ After your command it gave error to increase ram using limitGenomeGenerateRAM, then I increased the ram using limitGenomeGenerateRAM and finally it worked fine, thanks. $\endgroup$
    – Ammar Sabir Cheema
    Commented Jun 4, 2018 at 16:13
  • $\begingroup$ @AmmarSabirCheema, I am glad it finally works! Thanks for your feedback. $\endgroup$
    – benn
    Commented Jun 4, 2018 at 16:27
5
$\begingroup$

Try reducing the number of threads.

When multithreading the index creation (and other memory bound tasks), the memory usage increases linearly with the number of threads. If you required a 10 GiB working space work with a single thread, then the requirement for 10 threads would be at 100 GiB.

STAR is particularly memory hungry for the index creation to start with, and your genome is exceptionally large. I’ve had STAR require > 32 GiB of memory for the human genome, with a moderate number of threads. I’m not surprised that it fails with a 4× larger genome and many more threads: it would probably require at least 2× as much RAM as your machine has.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your answer but it didn't work. $\endgroup$
    – Ammar Sabir Cheema
    Commented Jun 4, 2018 at 9:26
  • $\begingroup$ I have again edited my post to add more details to my problem. $\endgroup$
    – Ammar Sabir Cheema
    Commented Jun 4, 2018 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.