An evaluation of the accuracy and speed of metagenome analysis tools

By Stinus Lindgreen, Karen L. Adair, Paul P. Gardner. Scientific Reports, 2015

309 citations recorded by the source corpus.

Metagenome studies are becoming increasingly widespread, yielding important insights into microbial communities covering diverse environments from terrestrial and aquatic ecosystems to human skin and gut. With the advent of high-throughput sequencing platforms, the use of large scale shotgun sequencing approaches is now commonplace. However, a thorough independent benchmark comparing state-of-the-art metagenome analysis tools is lacking. Here, we present a benchmark where the most widely used tools are tested on complex, realistic data sets. Our results clearly show that the most widely used tools are not necessarily the most accurate, that the most accurate tool is not necessarily the most time consuming and that there is a high degree of variability between available tools. These findings are important as the conclusions of any metagenomics study are affected by errors in the predicted community composition and functional capacity. Data sets and results are freely available from http://www.ucbioinformatics.org/metabenchmark.html

Subfields: Ecology, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/039f02cc985dfb205916aa76d762cdb392143442

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Ecology

In Molecular Biology

Citations in the corpus

Understanding the Promises and Hurdles of Metagenomic Next-Generation Sequencing as a Diagnostic Tool for Infectious Diseases (2017, background): “We refer the reader to other publications on this topic which summarize common terms and approaches used in DNA sequence analysis [10], the different NGS technologies [11-13], the various NGS methods, bioinformatics tools [14] and clinical applications [10, 15].”
Bracken: estimating species abundance in metagenomics data (2016, unknown): “These tools, however, employ different strategies for read-level classification which are not always as accurate and efficient as Kraken’s k -mer approach (Lindgreen, Adair & Gardner, 2016).”
MEGAN Community Edition - Interactive Exploration and Analysis of Large-Scale Microbiome Sequencing Data (2016, background): “See [22, 23] for two recent comparisons of the performance of different approaches.”
Fast and sensitive taxonomic classification for metagenomics with Kaiju (2016, methodology): “Wecompared the performance of Kaiju with the two k -mer-based programs Kraken and Clark, which performed best in speed and accuracy in a recent benchmark study 18 .”
Alignment-free sequence comparison: benefits, applications, and tools (2017, methodology): “The evaluation of the accuracy and speed of 14 widely used metagenome analysis tools [97] showed that Kraken and CLARK are top state-of-the-art tools with the highest speed, accuracy, and sensitivity (i.e., the fraction of reads that is correctly classified).”
Improved metagenomic analysis with Kraken 2 (2019, methodology): “We showed Kraken 2 ’ s accuracy is comparable to that of Kraken 1 and other competing tools, consistent with other studies [6,7].”
Benchmarking Metagenomics Tools for Taxonomic Classification. (2019, background): “Several groups have previously benchmarked metagenomic tools (Lindgreen et al., 2016; Mavromatis et al., 2007; McIntyre et al., 2017; Meyer et al., 2019; Sczyrba et al., 2017), but the continual introduction of newer tools requires ongoing evaluation to compare them against established tools.”