Performance of mutation pathogenicity prediction methods on missense variants

By J. Thusberg, Ayodeji Olatubosun, Mauno Vihinen. Human Mutation, 2011

566 citations recorded by the source corpus.

Single nucleotide polymorphisms (SNPs) are the most common form of genetic variation in humans. The number of SNPs identified in the human genome is growing rapidly, but attaining experimental knowledge about the possible disease association of variants is laborious and time‐consuming. Several computational methods have been developed for the classification of SNPs according to their predicted pathogenicity. In this study, we have evaluated the performance of nine widely used pathogenicity prediction methods available on the Internet. The evaluated methods were MutPred, nsSNPAnalyzer, Panther, PhD‐SNP, PolyPhen, PolyPhen2, SIFT, SNAP, and SNPs&GO. The methods were tested with a set of over 40,000 pathogenic and neutral variants. We also assessed whether the type of original or substituting amino acid residue, the structural class of the protein, or the structural environment of the amino acid substitution, had an effect on the prediction performance. The performances of the programs ranged from poor (MCC 0.19) to reasonably good (MCC 0.65), and the results from the programs correlated poorly. The overall best performing methods in this study were SNPs&GO and MutPred, with accuracies reaching 0.82 and 0.81, respectively. Hum Mutat 32:1–11, 2011. © 2011 Wiley‐Liss, Inc.

Subfields: Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/9218c7b13a9a3935236ace940496c2a98e295a55

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

Citations in the corpus

Ranking non-synonymous single nucleotide polymorphisms based on disease concepts (2014, background): “Discussion There is a plethora of computational prediction algorithms available to predict the functional consequences of nsSNPs [2].”
Comprehensive Analysis of Tissue-wide Gene Expression and Phenotype Data Reveals Tissues Affected in Rare Genetic Disorders (2017, background): “…approached through breaches of constraints on conservation and biochemistry (Adzhubei et al., 2013; Moreau and Tranchevent, 2012; Sim et al., 2012; Thusberg et al., 2011), protein interaction network properties (Barabási et al., 2011), guilt-by-association in multi-omic studies (Lee et al.,…”
Variation Interpretation Predictors: Principles, Types, Performance, and Choice (2016, methodology): “A dataset of over 40,000 variants, 19,335 positive and 21,170 negative cases, was used to test the performance of nine widely used tolerance methods [Thusberg et al., 2011].”
A new disease-specific machine learning approach for the prediction of cancer-causing missense variants (2011, methodology): “The missense SNPs without any evidences of association to disease in SwissVar and recently selected as negative cases [41] are indicated as neutral variants or polymorphisms.”
Genetic variations and diseases in UniProtKB/Swiss-Prot: The ins and outs of expert manual curation (2014, background): “Disease-linked variant p.Gln376Pro perturbs protein subcellular location. pathogenicity of missense variants [Thusberg et al., 2011; Capriotti et al., 2013; Shihab et al., 2013; Yates and Sternberg, 2013].”
Assessing the Pathogenicity of Insertion and Deletion Variants with the Variant Effect Scoring Tool (VEST‐Indel) (2015, background): “Given that poor specificity also limits the utility of methods aimed at assessing the pathogenicity of missense variants [Chan et al., 2007; Hicks et al., 2011; Thusberg et al., 2011; Shihab et al., 2013], the approach presented here might prove beneficial for variant classification in general.”
The Evaluation of Tools Used to Predict the Impact of Missense Variants Is Hindered by Two Types of Circularity (2015, methodology): “The VariBench dataset(benchmarkdatabase for variations) was created [Thusberg et al., 2011; Nair and Vihi-nenm 2013] to address the problem of type 1 circularity.”
PredictSNP: Robust and Accurate Consensus Classifier for Prediction of Disease-Related Mutations (2014, methodology): “The performances of individual tools observed with PredictSNP benchmark dataset were in good correspondence with a recent comprehensive evaluation of nine prediction methods by Thus-berg et al. [25].”
Evaluation of in silico algorithms for use with ACMG/AMP clinical variant interpretation guidelines (2017, result): “…interpreting large number of variants, full concordance, as suggested by the ACMG/AMP guidelines, is less likely to be achieved even when using only two algorithms, particularly for benign variants, consistent with earlier observation of poor correlations among predictors by Thusberg et al. [10].”
Standards and Guidelines for the Interpretation of Sequence Variants: A Joint Consensus Recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology (2015, background): “In general, most algorithms for missense variant prediction are 65-80% accurate when examining known disease variants.(16) Most tools also tend to have low specificity, resulting in over-prediction of missense changes as deleterious and are not as reliable at predicting missense variants with a mild”
Predicting the Functional, Molecular, and Phenotypic Consequences of Amino Acid Substitutions using Hidden Markov Models (2012, methodology): “The performance of FATHMM was compared against the performance of other computational prediction algorithms reported in two previously published reviews [Hicks et al., 2011; Thusberg et al., 2011].”
An integrative approach to predicting the functional effects of non-coding and coding sequence variation (2015, background): “There is a plethora of computational algorithms capable of predicting whether SNVs are deleterious (Thusberg et al., 2011).”
Comparison and integration of deleteriousness prediction methods for nonsynonymous SNVs in whole exome sequencing studies. (2015, methodology): “for TP variants VariBench dataset II (10,13) without”
Tools for Predicting the Functional Impact of Nonsynonymous Genetic Variation (2016, methodology): “Because of the reliance of sequence conservation methods on a multiple sequence alignment, it seems obvious that alignment quality would affect the prediction accuracy of these methods on benchmark tests (Ng and Henikoff 2006; Thusberg et al. 2011).”
SIFT web server: predicting effects of amino acid substitutions on proteins (2012, result): “Some of these algorithms have been directly compared to each other (25).”
InterVar: Clinical Interpretation of Genetic Variants by the 2015 ACMG-AMP Guidelines. (2017, background): “Many have a similar theoretical basis, but they also have known limitations, such as moderate accuracy, low specificity, and over-prediction.(17,18) Third and finally, public disease-specific and gene-specific databases, such as the Human Gene Mutation Database (HGMD),(19) ClinVar,(20) and various l”
WS-SNPs&GO: a web server for predicting the deleterious effect of human protein variants using functional annotation (2013, methodology): “…is a good alternative to well-established tools like SIFT [18] and PolyPhen [23] since it returns high quality predictions as shown in previous works [19,20,40] and confirmed in the 2011 edition of the Critical Assessment of Genome Interpretation experiments (http://genomeinterpretation.org/).”
Collective judgment predicts disease-associated single nucleotide variants (2013, background): “Whatever the reason, however, most of the existing computational tools study the effects of nsSNVs specifically [3-6].”