Open Access Research Article Artificial and Natural Duplicates in Pyrosequencing Reads of Metagenomic Data

By Beifang Niu, Limin Fu, Shulei Sun, Weizhong Li

290 citations recorded by the source corpus.

BackgroundArtificial duplicates from pyrosequencing reads may lead to incorrect interpretation of the abundance of species and genes in metagenomic studies. Duplicated reads were filtered out in many metagenomic projects. However, since the duplicated reads observed in a pyrosequencing run also include natural (non-artificial) duplicates, simply removing all duplicates may also cause underestimation of abundance associated with natural duplicates.ResultsWe implemented a method for identification of exact and nearly identical duplicates from pyrosequencing reads. This method performs an all-against-all sequence comparison and clusters the duplicates into groups using an algorithm modified from our previous sequence clustering method cd-hit. This method can process a typical dataset in ~10 minutes; it also provides a consensus sequence for each group of duplicates. We applied this method to the underlying raw reads of 39 genomic projects and 10 metagenomic projects that utilized pyrosequencing technique. We compared the occurrences of the duplicates identified by our method and the natural duplicates made by independent simulations. We observed that the duplicates, including both artificial and natural duplicates, make up 4-44% of reads. The number of natural duplicates highly correlates with the samples' read density (number of reads divided by genome size). For high-complexity metagenomic samples lacking dominant species, natural duplicates only make up <1% of all duplicates. But for some other samples like transcriptomic samples, majority of the observed duplicates might be natural duplicates.ConclusionsOur method is available from http://cd-hit.org as a downloadable program and a web server. It is important not only to identify the duplicates from metagenomic datasets but also to distinguish whether they are artificial or natural duplicates. We provide a tool to estimate the number of natural duplicates according to user-defined sample types, so users can decide whether to retain or remove duplicates in their projects.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

Metagenomics - a guide from sampling to data analysis (2012, background): “Understanding the amount of replicate sequences is crucial for the data quality of sequencing runs, and replicates can be identified and filtered out with bioinformatics tools [22,23].”
Shining a Light on Dark Sequencing: Characterising Errors in Ion Torrent PGM Data (2013, background): “This ‘replicate bias’ is generally attributed to issues within the emulsion PCR (emPCR) step [9,10].”
Community cyberinfrastructure for Advanced Microbial Ecology Research and Analysis: the CAMERA resource (2010, background): “The 454 Duplicate Clustering identifies exact and near identical duplicates to remove sequencing artifacts (15).”
VIROME: a standard operating procedure for analysis of viral metagenome sequences (2012, methodology): “Requirements The VIROME analysis pipeline relies on three subject protein sequence databases, five annotated databases, the UniVec database, and CD-Hit 454 [17].”
The Pacific Ocean Virome (POV): A Marine Viral Metagenomic Dataset and Associated Protein Clusters for Quantitative Viral Ecology (2013, methodology): “5 and requiring a percent identity of 100% across the smallest contig [68].”
WebMGA: a customizable web server for fast metagenomic sequence analysis (2011, unknown): “(cid:129) Sequence clustering has 4 tools: CD-HIT-EST, CD-HIT, H-CD-HIT [17-20] and CD-HIT-454 [31].”
Ultrafast clustering algorithms for metagenomic sequence analysis (2012, methodology): “Later, CD-HIT-454 [13] was introduced by reengineering CD-HIT-EST.”
Hybrid error correction and de novo assembly of single-molecule sequencing reads (2012, background): “First, they require amplification of source DNA before sequencing, leading to amplification artifacts 4 and biased coverage of the genome related to the chemical-physical properties of the DNA 5 .”
The Amborella Genome and the Evolution of Flowering Plants (2013, methodology): “, Phylogeny and diversification of B-function MADS-box genes in angiosperms: evolutionary and functional implications of a 260-million-year-old duplication, Am J Bot 91, 2102–2118 (2004).”
CD-HIT: accelerated for clustering the next-generation sequencing data (2012, background): “…ranging from non-redundant dataset creation (Suzek, et al., 2007), protein family classifications (Yooseph, et al., 2008), artifact identification (Niu, et al., 2010), metagenomics annotation (Sun, et al., 2011), RNA analysis (Loong and Mishra, 2007), to various prediction studies (Rubinstein…”