Subsampled open-reference clustering creates consistent, comprehensive OTU definitions and scales to billions of sequences

By J. Rideout, Yan He, Jose A Navas-Molina, William Walters, L. Ursell, Sean M. Gibbons, John Chase, Daniel McDonald, Antonio Gonzalez, A. Robbins-Pianka and 6 more. PeerJ, 2014

518 citations recorded by the source corpus.

We present a performance-optimized algorithm, subsampled open-reference OTU picking, for assigning marker gene (e.g., 16S rRNA) sequences generated on next-generation sequencing platforms to operational taxonomic units (OTUs) for microbial community analysis. This algorithm provides benefits over de novo OTU picking (clustering can be performed largely in parallel, reducing runtime) and closed-reference OTU picking (all reads are clustered, not only those that match a reference database sequence with high similarity). Because more of our algorithm can be run in parallel relative to “classic” open-reference OTU picking, it makes open-reference OTU picking tractable on massive amplicon sequence data sets (though on smaller data sets, “classic” open-reference OTU clustering is often faster). We illustrate that here by applying it to the first 15,000 samples sequenced for the Earth Microbiome Project (1.3 billion V4 16S rRNA amplicons). To the best of our knowledge, this is the largest OTU picking run ever performed, and we estimate that our new algorithm runs in less than 1/5 the time than would be required of “classic” open reference OTU picking. We show that subsampled open-reference OTU picking yields results that are highly correlated with those generated by “classic” open-reference OTU picking through comparisons on three well-studied datasets. An implementation of this algorithm is provided in the popular QIIME software package, which uses uclust for read clustering. All analyses were performed using QIIME’s uclust wrappers, though we provide details (aided by the open-source code in our GitHub repository) that will allow implementation of subsampled open-reference OTU picking independently of QIIME (e.g., in a compiled programming language, where runtimes should be further reduced). Our analyses should generalize to other implementations of these OTU picking algorithms. Finally, we present a comparison of parameter settings in QIIME’s OTU picking workflows and make recommendations on settings for these free parameters to optimize runtime without reducing the quality of the results. These optimized parameters can vastly decrease the runtime of uclust-based OTU picking in QIIME.

View Paper: https://www.semanticscholar.org/paper/129b705776df6b4383fe0cc6f1ff6fc6d6ff1a40

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Swarm v2: highly-scalable and high-resolution amplicon clustering (2015, methodology): “Because of this scalability problem, Rideout et al. (2014) proposed using a mixed clustering approach with an initial closed-reference clustering that compares the amplicons to what is known in taxonomic reference databases to capture most of the data, followed by a de novo clustering with the remai”
QIIME 2 Enables Comprehensive End‐to‐End Analysis of Diverse Microbiome Data and Comparative Studies with Publicly Available Data (2020, methodology): “The shorthand “artifact” in the per-sample Deblur statistics denotes artifactual sequences (i.e., those erroneously generated as byproducts of the PCR and DNA sequencing process), not a QIIME 2 artifact (i.e., a valid data product of QIIME 2).”
Updating the 97% identity threshold for 16S ribosomal RNA OTUs (2017, methodology): “ZOTUs have the additional advantage of being directly comparable between datasets without re-clustering (i.e., ZOTUs are stable as defined by (Rideout et al., 2014)), providing that the same gene segment is compared.”
Exact sequence variants should replace operational taxonomic units in marker-gene data analysis (2017, methodology): “De novo OTUs lack consistent labels so all data must be pooled for assignment, causing computational costs to scale quadratically with total sequencing effort and rendering common de novo methods prohibitively costly as study sizes increase (Rideout et al, 2014; Mahe et al, 2015).”
Partial restoration of the microbiota of cesarean-born infants via vaginal microbial transfer (2016, unknown): “Quality-filtered reads were clustered into Operational Taxonomic Units (OTUs) using an open-reference algorithm algorithm 21 and Greengenes v13_8 as a reference set 22 .”
OptiClust: Improved method for assigning amplicon-based sequence data to operational taxonomic units (2016, methodology): “It has become common for studies to include sequencing millions of 16S rRNA gene sequences 32 across hundreds of samples (3, 4).”
Comparing bioinformatic pipelines for microbial 16S rRNA amplicon sequencing (2020, methodology): “This script implements the latest QIIME open reference OTU clustering [20].”
The Wheat Microbiome Under Four Management Strategies, and Potential for Endophytes in Disease Protection (2017, methodology): “To determine within-sample diversity, alpha diversity statistics were calculated with the ‘phyloseq’ package (McMurdie and Holmes 2013; Rideout et al. 2014), significance was tested by analysis of variance (ANOVA) and Tukey’s honest significant difference in the R statistical computing environment.”