Low concordance of multiple variant-calling pipelines: practical implications for exome and genome sequencing

By G. Lyon, Ph, A. Moore, Jeffrey J Rope, Lynn Swensen, Mark Jorde, Yandell, Reid J. Robison, Jason O 'rawe, Yiyang Wu and 18 more. Genome Medicine, 2013

470 citations recorded by the source corpus.

BackgroundTo facilitate the clinical implementation of genomic medicine by next-generation sequencing, it will be critically important to obtain accurate and consistent variant calls on personal genomes. Multiple software tools for variant calling are available, but it is unclear how comparable these tools are or what their relative merits in real-world scenarios might be.MethodsWe sequenced 15 exomes from four families using commercial kits (Illumina HiSeq 2000 platform and Agilent SureSelect version 2 capture kit), with approximately 120X mean coverage. We analyzed the raw data using near-default parameters with five different alignment and variant-calling pipelines (SOAP, BWA-GATK, BWA-SNVer, GNUMAP, and BWA-SAMtools). We additionally sequenced a single whole genome using the sequencing and analysis pipeline from Complete Genomics (CG), with 95% of the exome region being covered by 20 or more reads per base. Finally, we validated 919 single-nucleotide variations (SNVs) and 841 insertions and deletions (indels), including similar fractions of GATK-only, SOAP-only, and shared calls, on the MiSeq platform by amplicon sequencing with approximately 5000X mean coverage.ResultsSNV concordance between five Illumina pipelines across all 15 exomes was 57.4%, while 0.5 to 5.1% of variants were called as unique to each pipeline. Indel concordance was only 26.8% between three indel-calling pipelines, even after left-normalizing and intervalizing genomic coordinates by 20 base pairs. There were 11% of CG variants falling within targeted regions in exome sequencing that were not called by any of the Illumina-based exome analysis pipelines. Based on targeted amplicon sequencing on the MiSeq platform, 97.1%, 60.2%, and 99.1% of the GATK-only, SOAP-only and shared SNVs could be validated, but only 54.0%, 44.6%, and 78.1% of the GATK-only, SOAP-only and shared indels could be validated. Additionally, our analysis of two families (one with four individuals and the other with seven), demonstrated additional accuracy gained in variant discovery by having access to genetic data from a multi-generational family.ConclusionsOur results suggest that more caution should be exercised in genomic medicine settings when analyzing individual genomes, including interpreting positive and negative findings with scrutiny, especially for indels. We advocate for renewed collection and sequencing of multi-generational families to increase the overall accuracy of whole genomes.

Subfields: Cancer Research, Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/0cbc1ee4c3056a8e9f43bc027aaea79980875dd8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cancer Research

In Genetics

In Molecular Biology

Citations in the corpus

Toward better understanding of artifacts in variant calling from high-coverage samples (2014, result): “…variant calls from different pipelines, or by comparing calls to variants ascertained with array genotyping or in another study (Clark et al., 2011; Li, 2012; Lam et al., 2012a,b; Boland et al., 2013; Liu et al., 2013; Goode et al., 2013; O’Rawe et al., 2013; Zook et al., 2014; Cheng et al., 2014).”
Integrating mapping-, assembly- and haplotype-based approaches for calling variants in clinical sequencing applications (2014, background): “However, despite substantial progress in algorithm development 1–6 , the processing of these data into high-quality variant calls remains challenging 7 .”
A framework for incorporating evolutionary genomics into biodiversity conservation and management (2015, background): “Identifying indels from high-throughput sequencing data remains a difficult bioinformatic problem [77], and identifying large indels in RRS data is especially difficult because only a small percentage of the genome is sequenced.”
Next-generation diagnostics and disease-gene discovery with the Exomiser (2015, background): “…any of the previous steps in the pipeline here, although practitioners need to be aware of the fact that different pipelines will call divergent sets of variants, with obvious consequences for downstream analysis 56 , and that the provenance of steps needs to be carefully considered and tracked.”
A SPECTRAL APPROACH INTEGRATING FUNCTIONAL GENOMIC ANNOTATIONS FOR CODING AND NONCODING VARIANTS (2016, background): “However it is currently difficult to detect indels with high accuracy from short read sequence data [44, 45].”
A reference data set of 5.4 million phased human variants validated by genetic inheritance from sequencing a three-generation 17-member pedigree (2016, unknown): “A second approach is to compare technical and/or informatic replicates of a dataset (Lam et al. 2012; O'Rawe et al. 2013; Zook et al. 2014).”
Variation Interpretation Predictors: Principles, Types, Performance, and Choice (2016, background): “Bear in mind that the interpretations of variation callers do not always agree [O’Rawe et al., 2013].”
An international effort towards developing standards for best practices in analysis, interpretation and reporting of clinical genome sequencing results in the CLARITY Challenge (2014, background): “While GATK and SAMtools are the most popular variant callers used today and reporte d in his survey, their relative performance has been shown to vary with the sequencing dept h [35,36], and direct comparison of variant calls resulting from parallel analysis of the same raw data by different variant”