Waste Not, Want Not: Why Rarefying Microbiome Data Is Inadmissible

By Paul J. McMurdie, Susan P. Holmes. PLoS Comput. Biol., 2013

2,595 citations recorded by the source corpus.

Current practice in the normalization of microbiome count data is inefficient in the statistical sense. For apparently historical reasons, the common approach is either to use simple proportions (which does not address heteroscedasticity) or to use rarefying of counts, even though both of these approaches are inappropriate for detection of differentially abundant species. Well-established statistical theory is available that simultaneously accounts for library size differences and biological variability using an appropriate mixture model. Moreover, specific implementations for DNA sequencing read count data (based on a Negative Binomial model for instance) are already available in RNA-Seq focused R packages such as edgeR and DESeq. Here we summarize the supporting statistical theory and use simulations and empirical data to demonstrate substantial improvements provided by a relevant mixture model framework over simple proportions or rarefying. We show how both proportions and rarefied counts result in a high rate of false positives in tests for species that are differentially abundant across sample classes. Regarding microbiome sample-wise clustering, we also show that the rarefying procedure often discards samples that can be accurately clustered by alternative methods. We further compare different Negative Binomial methods with a recently-described zero-inflated Gaussian mixture, implemented in a package called metagenomeSeq. We find that metagenomeSeq performs well when there is an adequate number of biological replicates, but it nevertheless tends toward a higher false positive rate. Based on these results and well-established statistical theory, we advocate that investigators avoid rarefying altogether. We have provided microbiome-specific extensions to these tools in the R package, phyloseq.

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/702b5b7832f439d024d8fda91358a6bb948f6eee

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Effects of plant community history, soil legacy and plant diversity on soil microbial communities (2020, neutral): “Sequencing data were not rarefied (McMurdie & Holmes, 2014).”
Normalization and microbial differential abundance strategies depend upon data characteristics (2017, methodology): “Recent work in this area [18] addresses the performance of parametric normalization and differential abundance testing approaches for microbial ecology studies, but it is primarily focused on estimating proportions.”
NetCoMi: network construction and comparison for microbiome data in R (2020, positive): “Normalization techniques are required to make read counts comparable across different samples [47,48] (step 1c in Figure 1).”
Sparse and Compositionally Robust Inference of Microbial Ecological Networks (2014, methodology): “Most methods test each taxon independently one-at-a-time (see [56] and references therein) even though taxa are actually highly correlated and thought to ecologically interact.”
Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2 (2014, neutral): “, [43]), ribosome profiling [44] and CRISPR/Cas-library assays [45].”
Conducting a Microbiome Study (2014, background): “Recent large-scale analyses provide examples of how to apply multivariate statistics to 16S rRNA sequence data (Bokulich et al., 2013b; Lundberg et al., 2012; McMurdie and Holmes, 2013; Peiffer et al., 2013).”
Where less may be more: how the rare biosphere pulls ecosystems strings (2017, background): “Recent studies have shown that low-abundance species should be considered full members of microbial communities and not treated as analytical annoyances (Nipperess and Matsen, 2013; McMurdie and Holmes, 2014; Lynch and Neufeld, 2015).”
A Novel Sparse Compositional Technique Reveals Microbial Perturbations (2019, background): “However, because microbiome sequencing data are represented as counts (17), the data are strictly positive and skewed toward zero, which confounds PCA.”
Assembly and ecological function of the root microbiome across angiosperm plant species (2018, methodology): “For downstream composition analyses, we performed proportional abundance normalization (relative abundance) on this common set of ASVs, where the sequencing reads for an ASV in a given sample were divided by the total number of sequencing reads in that sample (66).”
Evaluation of Filtration and DNA Extraction Methods for Environmental DNA Biodiversity Assessments across Multiple Trophic Levels (2017, methodology): “This method accounts for differential sample depth (correcting for uneven numbers of reads per tag) and is appropriate for normalizing high-variance data sets from high-throughput sequencing (McMurdie and Holmes, 2014).”
The Wheat Microbiome Under Four Management Strategies, and Potential for Endophytes in Disease Protection (2017, methodology): “Samples were normalized with variance-stabilizing normalization and significant OTUs were identified using the ‘DESeq2’ package (Love et al. 2014; McMurdie and Holmes 2014).”
Gut bacteria from multiple sclerosis patients modulate human T cells and exacerbate symptoms in mouse models (2017, methodology): “Instead, OTU abundances were normalized using variance-stabilizing transformation and taxa distributions were compared using Wald negative binomial test from R software package DESeq2 as described previously (4, 7) with Benjamini-Hochberg correction for multiple comparisons.”
Transient leaf endophytes are the most active fungi in 1-year-old beech leaf litter (2017, background): “07) between sequencing depth and OTU richness in these categories tolerable and therefore refrained from excessive data manipulation by rarefying (McMurdie and Holmes 2014).”