7
$\begingroup$

I am running Picard MarkDuplicates with the following parameters below. On the file described, it takes about 41.6Gb of RAM memory and about 20-25 minutes to compute (only uses 1 core AFAICS).

java -Xmx54890m -jar picard.jar MarkDuplicates \
    I=/home/dnanexus/in/sorted_bam/TST73-86-2IC_S16_L00.bml.GRCh38.karyo.bam \
    O=./TST73-86-2IC_S16_L00.bml.GRCh38.karyo.deduplicated.bam \
    M=./TST73-86-2IC_S16_L00.bml.GRCh38.karyo.duplication_metrics \
    CREATE_INDEX=true VALIDATION_STRINGENCY=SILENT REMOVE_DUPLICATES=true

What equivalent free/open source software (samtools?, sambamba?) can I use to obtain the same deduplicated bam file but at a lower cost, considering the instance used here has 8 cores but only one of them is being used, and has about 68Gb of RAM out of which it's using 46Gb odd at peak.

I think previous versions of samtools/sambamba didn't behave the same way as Picard MarkDuplicates with the parameters above, but I would like to know which recent versions would give me the same output.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ I assume you only want free (in terms of price, at least) software, right? Sentieon have a tool for this that is orders of magnitude faster (and I would guess uses less RAM but I'd have to check) but you need to pay for it. $\endgroup$
    – terdon ♦
    Commented Dec 11, 2017 at 15:58
  • $\begingroup$ Correct, open source. $\endgroup$
    – 719016
    Commented Dec 11, 2017 at 16:05
  • $\begingroup$ I done no benchmark, but I guess samblaster is bit faster (by my subjective perception of time). $\endgroup$
    – Kamil S Jaron
    Commented Dec 11, 2017 at 20:24
  • $\begingroup$ You should specify a smaller "-Xmx". MarkDuplicate doesn't need that much memory, but seeing a large "-Xmx", java will use extra. $\endgroup$
    – user172818
    Commented Dec 12, 2017 at 15:21

1 Answer 1

Reset to default
2
$\begingroup$

As you suggested, sambamba is faster at marking duplicates than picard (it's also multithreaded). Recent versions of samtools have a rewritten duplicate marking algorithm, though I doubt it'll be as quick as sambamba. 46GB of RAM seems excessive for marking duplicates unless you're having it store the whole file in memory.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Does sambamba with default parameters equate to Picard MarkDuplicates? $\endgroup$
    – 719016
    Commented Dec 12, 2017 at 21:50
  • $\begingroup$ I'm not sure that any tool exactly equates (there are a lot of edge cases in duplicate marking and picard misses some things), but it's a sufficient replacement. Sambamba is also used for this purpose by a number of international consortia (I think either DEEP or IHEC, if I remember correctly). $\endgroup$
    – Devon Ryan
    Commented Dec 12, 2017 at 23:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.