An open resource for accurately benchmarking small variant and reference calls

By Justin M. Zook, Jennifer McDaniel, Nathan D. Olson, Justin Wagner, H. Parikh, Haynes Heaton, S. Irvine, Len Trigg, Rebecca M. Truty, Cory Y. McLean and 4 more. Nature Biotechnology, 2019

303 citations recorded by the source corpus.

Benchmark small variant calls are required for developing, optimizing and assessing the performance of sequencing and bioinformatics methods. Here, as part of the Genome in a Bottle (GIAB) Consortium, we apply a reproducible, cloud-based pipeline to integrate multiple short- and linked-read sequencing datasets and provide benchmark calls for human genomes. We generate benchmark calls for one previously analyzed GIAB sample, as well as six genomes from the Personal Genome Project. These new genomes have broad, open consent, making this a ‘first of its kind’ resource that is available to the community for multiple downstream applications. We produce 17% more benchmark single nucleotide variations, 176% more indels and 12% larger benchmark regions than previously published GIAB benchmarks. We demonstrate that this benchmark reliably identifies errors in existing callsets and highlight challenges in interpreting performance metrics when using benchmarks that are not perfect or comprehensive. Finally, we identify strengths and weaknesses of callsets by stratifying performance according to variant type and genome context. Genome in a Bottle Consortium presents a high-confidence dataset for benchmarking small variant calls in human genomes.

Subfields: Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/f9830fecb0a4862aa458dae02721b613cce0e992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

Citations in the corpus

Accurate circular consensus long-read sequencing improves variant detection and assembly of a human genome (2019, methodology): “Evaluated against the GIAB benchmark 26 , genome-wide precision for SNVs is 99.468% and recall is 99.559%.”
Best practices for variant calling in clinical sequencing (2020, methodology): “The GIAB dataset has been continually improved with the addition of data from multiple shortread and linked-read sequencing datasets and the expansion of the reference from one sample to seven [69].”
Haplotype-resolved diverse human genomes and integrated analysis of structural variation (2021, methodology): “We additionally sequenced three previously studied child samples, completing three parent-child trios, and we included for analysis publicly available sequencing data for two samples, NA12878 and HG002/NA24385, that were generated as part of the Genome in a Bottle effort ( 17 ).”
Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes (2019, background): “Currently, five human genomes are available as NIST Reference Materials with benchmark small variant and reference calls for approximately 90% of GRCh37 and GRCh38 5,7,24 .”
HiCanu: accurate assembly of segmental duplications, satellites, and allelic variants from high-fidelity long reads (2020, methodology): “HG002 and parent Illumina data are available from GIAB (Zook et al. 2016) at GitHub (https://github.com/genome-in-a-bottle/ giab_data_indexes), we only used the 2×250 data sets.”