3
$\begingroup$

This page is claimed to contain a gold standard benchmark for viral genome assembly. https://github.com/cbg-ethz/5-virus-mix

The claim is here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/

Baaijens et al. De novo assembly of viral quasispecies using overlap graphs Genome Research 2017 May; 27: 835–848.

"This data set was recently presented as a gold standard benchmark (Di Giallonardo et al. 2014)"

But at (Di Giallonardo et al. 2014) I can only find that mention:

"We previously estimated the frequencies of the virus strains in the 5-virus-mix by amplifying the protease gene using single-genome amplification (SGA), the current gold standard for studying diversity of virus populations".

That doesn't seem to mean precisely that the dataset is a gold standard benchmark for genome assembly but, as I understand, that SGA is a gold standard for something else.

My question is: is it? And who decided it is? Is still valid? Is there an official list of results somewhere for this benchmark set?

I tried to contact the owner of the page but still no answer.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Anyone can declare anything to be a "gold-standard", the term is meaningless. All if actually means is, "we think this is the test that everything else should be benchmarked against".

Improve this answer
$\endgroup$
1
$\begingroup$

Thanks for raising the issue, it is certainly topical. The question is whether the data set they unit tested is reliable.

I think the issue they are trying get to is mosaic artefacts in the sequencing process and assembly thereon. This would be particularly true in a situation of de novo assembly which is what the authors are targeting. There are some slick methodologies for virus sequencing, which have been around for a while now, so in any case a 'gold-standard' would be outdated at present.

I do have to agree with you that the language particularly for the journal certainly could have phrased better. The data set was 2014 and genome circularisation (its a virus small genomes) to remove the otherwise high SNP error rate would have been around in 2017 when this was published.

I fully understand why the authors choose this language. A professional editor, not affiliated with the journal (without up to date technical insight), would likely recommend the same language.

I guess the question would be whether verification is achieved on other virus data sets. There is no shortage of them at present and its certainly an interesting tool.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Well, I was not asking that because of any suspicion about their results. My software (s-aligner) gets better results for this benchmark set and I just wanted to know how relevant is that. It also outperforms other software in many other benchmarks. $\endgroup$
    – juanjo75es
    Commented Feb 28, 2022 at 21:27
  • $\begingroup$ Thanks @juanjo75es this is a good subject to explore in a question. The algorithms are pivotal to understanding viral diversity. I put this as a separate question and notify you. I'll explain what I understand and why within the question. $\endgroup$
    – M__ ♦
    Commented Mar 1, 2022 at 15:20
  • $\begingroup$ @juanjo75es I have raised the matter in the following question, bioinformatics.stackexchange.com/questions/18747/… $\endgroup$
    – M__ ♦
    Commented Mar 9, 2022 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.