A generic method for assignment of reliability scores applied to solvent accessibility predictions

By Bent Petersen, Thomas Nordahl Petersen, P. Andersen, Morten Nielsen, Claus Lundegaard. 2009

646 citations recorded by the source corpus.

BackgroundEstimation of the reliability of specific real value predictions is nontrivial and the efficacy of this is often questionable. It is important to know if you can trust a given prediction and therefore the best methods associate a prediction with a reliability score or index. For discrete qualitative predictions, the reliability is conventionally estimated as the difference between output scores of selected classes. Such an approach is not feasible for methods that predict a biological feature as a single real value rather than a classification. As a solution to this challenge, we have implemented a method that predicts the relative surface accessibility of an amino acid and simultaneously predicts the reliability for each prediction, in the form of a Z-score.ResultsAn ensemble of artificial neural networks has been trained on a set of experimentally solved protein structures to predict the relative exposure of the amino acids. The method assigns a reliability score to each surface accessibility prediction as an inherent part of the training process. This is in contrast to the most commonly used procedures where reliabilities are obtained by post-processing the output.ConclusionThe performance of the neural networks was evaluated on a commonly used set of sequences known as the CB513 set. An overall Pearson's correlation coefficient of 0.72 was obtained, which is comparable to the performance of the currently best public available method, Real-SPINE. Both methods associate a reliability score with the individual predictions. However, our implementation of reliability scores in the form of a Z-score is shown to be the more informative measure for discriminating good predictions from bad ones in the entire range from completely buried to fully exposed amino acids. This is evident when comparing the Pearson's correlation coefficient for the upper 20% of predictions sorted according to reliability. For this subset, values of 0.79 and 0.74 are obtained using our and the compared method, respectively. This tendency is true for any selected subset.

Subfields: Computational Theory and Mathematics, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Molecular Biology

Citations in the corpus

NetSurfP-2.0: improved prediction of protein structural features by integrated deep learning (2018, methodology): “0 (10) is a tool published in 2009 for prediction of solvent accessibility and secondary structure using a feed-forward neural network architecture.”
Improved Contact Predictions Using the Recognition of Protein Like Contact Patterns (2014, methodology): “Best performance was obtained using window size of 9 residues, encoding relative surface accessibility, Z-fit score (reliability value), plus probabilities for three secondary structure classes as predicted by NetSurfP (Helix, Strand and Coil).”
Maximum Allowed Solvent Accessibilites of Residues in Proteins (2012, methodology): “computational methods to predict RSA from protein primary and/or secondary structure [14–20].”
A Proteome-wide, Quantitative Survey of In Vivo Ubiquitylation Sites Reveals Widespread Regulatory Roles (2011, methodology): “Secondary structure analysis was performed using NetSurfP (32).”
CPHmodels-3.0—remote homology modeling using structure-guided sequence profiles (2010, methodology): “Prediction of local protein structure features can also improve when sequence proﬁles are used to represent the protein sequences (5–7).”
SuSPect: Enhanced Prediction of Single Amino Acid Variant (SAV) Phenotype Using Network Features (2014, neutral): “NetSurfP was used to predict secondary structure and RSA based on the results of a PSI-BLAST search [42].”
A highly conserved program of neuronal microexons is misregulated in autistic brains (2014, methodology): “1 (Petersen et al., 2009) and default assignments of accessibility were used.”