15
$\begingroup$

The industry standard for aligning short reads seems to be bwa-mem. However, in my tests I have seen that using bwa backtrack (bwa-aln + bwa-sampe + bwa-samse) performs better. It is slightly slower, but gives significantly better results in terms of both sensitivity and specificity. I have tested it using the genome in a bottle data and public samples (NA12878 and NA12877 among others) and found that backtrack consistently outperformed bwa-mem.

So why is bwa-mem the standard? Am I wrong and other tests have shown the opposite? I don't really see how since I tested using the most common datasets and validation data. Is it that the slight increase in efficiency outweighs the decrease in performance?

The only other explanation I can see is that bwa backtrack is designed specifically for Illumina reads and all my tests have been on Illumina data. Is it just that bwa-mem is "sequencer agnostic"? So that we can use the same algorithm irrespective of what sequencing platform is used? In that case, it makes sense to use backtrack for if we only deal with Illumina data and mem if we can have different sequencers. But, if so, seeing as Illumina is so widespread, why isn't backtrack used more often on Illumina data? I feel I must be missing something.

Improve this question
$\endgroup$
10
  • 6
    $\begingroup$ Heng Li would be the best person to respond to this.... Also it might be worth citing existing benchmarks or giving details of those which you have run. $\endgroup$
    – Matt Bashton
    Jun 3, 2017 at 14:03
  • 2
    $\begingroup$ There is always the bwa mem paper too arxiv.org/pdf/1303.3997.pdf $\endgroup$
    – Matt Bashton
    Jun 3, 2017 at 14:06
  • 1
    $\begingroup$ @MatthewBashton yes, I know. Unfortunately, I don't have them in a form that's easy to show and I ran them more than a year ago now so they might not even be relevant anymore. I am hoping that others have seen similar issues and know why one would be preferred over the other. Or that a review article has been published comparing them and my pubmed-fu just failed me. If not, I guess I'll rerun them myself, look into it more deeply and post a new question. $\endgroup$
    – terdon ♦
    Jun 3, 2017 at 14:06
  • 1
    $\begingroup$ What's your read length? bio-bwa.sourceforge.net recommends mem as best for >70 bp. It would be interesting to see a reproducible benchmark if you've found otherwise $\endgroup$
    – blmoore
    Jun 3, 2017 at 17:54
  • 1
    $\begingroup$ Hi @terdon; our sequencing core team has looked at BWA-MEM vs backtrack for several data sets and have consistently found that MEM is both faster and more accurate than backtrack. We've also found that if you have significant quality drop-offs at the tail backtrack's performance suffers heavily... and that's another advantage of MEM: you don't need quality trimming, where backtrack needs reads to be mapped in full-length, which we have observed independently of, and in addition to Heng Li (see this thread). $\endgroup$
    – Wyatt McDonnell
    Jun 3, 2017 at 18:30

1 Answer 1

Reset to default
7
$\begingroup$

bwa mem is newer, faster, and [should be] more accurate, particularly for longer reads.

From the bwa man page (presumably in Heng Li's own words):

BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms: BWA-backtrack, BWA-SW and BWA-MEM. The first algorithm is designed for Illumina sequence reads up to 100bp, while the rest two for longer sequences ranged from 70bp to 1Mbp. BWA-MEM and BWA-SW share similar features such as long-read support and split alignment, but BWA-MEM, which is the latest, is generally recommended for high-quality queries as it is faster and more accurate. BWA-MEM also has better performance than BWA-backtrack for 70-100bp Illumina reads.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Yes, I know that's what he claims. That's just not what I've seen. Do you have any benchmarks or a review article or anything demonstrating that bwa-mem is indeed more accurate? $\endgroup$
    – terdon ♦
    Jun 4, 2017 at 10:57
  • $\begingroup$ Heng Li's paper on BWA is here. It compares the accuracy of BWA-MEM with BWA-SW, and demonstrates that BWA-MEM has a higher number of mapped reads for the same number of mismapped reads (see Fig. 1). $\endgroup$
    – gringer ♦
    Jun 4, 2017 at 11:05
  • 1
    $\begingroup$ Brian Bushnell seems to think that BWA-MEM is more accurate than BWA-backtrack. $\endgroup$
    – gringer ♦
    Jun 4, 2017 at 11:34
  • 6
    $\begingroup$ To be fair, you also haven't given any evidence supporting this. It would be great if you could add a graph, or other results into your question (preferably with a downloadable read/reference set). $\endgroup$
    – gringer ♦
    Jun 4, 2017 at 21:10
  • 2
    $\begingroup$ Yes indeed. I should show my data. To be honest, I did this more than a year ago and just don't have them anymore. It is absolutely possible that I'm wrong and there is no reason whatsoever for anyone to believe me just because I say so. I was hoping someone could answer with benchmarks of their own or, even better, with a peer-reviewed article showing comparison data. $\endgroup$
    – terdon ♦
    Jun 4, 2017 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.