Evaluation of disorder predictions in CASP9

By Bohdan Monastyrskyy, Krzysztof Fidelis, John Moult, Anna Tramontano, Andriy Kryshtafovych. 2011

121 citations recorded by the source corpus.

Lack of stable three‐dimensional structure, or intrinsic disorder, is a common phenomenon in proteins. Naturally, unstructured regions are proven to be essential for carrying function by many proteins, and therefore identification of such regions is an important issue. CASP has been assessing the state of the art in predicting disorder regions from amino acid sequence since 2002. Here, we present the results of the evaluation of the disorder predictions submitted to CASP9. The assessment is based on the evaluation measures and procedures used in previous CASPs. The balanced accuracy and the Matthews correlation coefficient were chosen as basic measures for evaluating the correctness of binary classifications. The area under the receiver operating characteristic curve was the measure of choice for evaluating probability‐based predictions of disorder. The CASP9 methods are shown to perform slightly better than the CASP7 methods but not better than the methods in CASP8. It was also shown that capability of most CASP9 methods to predict disorder decreases with increasing minimum disorder segment length. Proteins 2011;. © 2011 Wiley‐Liss, Inc.

Subfields: Materials Chemistry, Molecular Biology, Spectroscopy

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Spectroscopy

Citations in the corpus

New encouraging developments in contact prediction: assessment of the CASP11 results (2015, methodology): “33 For the bootstrapping, we randomly sampled (with replacement) the list of targets predicted by each group, and recalculated the evaluation scores on the resampled target sets.”
RaptorX-Property: a web server for protein structure property prediction (2016, background): “Following the definition in (23), we label a residue as disordered (denoted as ‘*’) if it is in a segment of more than three residues missing atomic coordinates in the Xray structure.”
DISOPRED3: precise disordered region predictions with annotated protein-binding activity (2014, background): “This was done for practical reasons, because a nearest neighbor classifier needs no training, and so it is easy to incorporate very up-to-date data in the prediction process, whereas the other classifiers require a great deal of time and effort to train on new data.”
DeepCNF-D: Predicting Protein Order/Disorder Regions by Weighted Deep Convolutional Neural Fields (2015, methodology): “We further trained our weighted DeepCNF model using UniProt90 dataset for release purpose and evaluated it with two publicly available CASP datasets, i.e. , CASP9 [25] and CASP10 [15].”
Disorder Prediction Methods, Their Applicability to Different Protein Targets and Their Usefulness for Guiding Experimental Studies (2015, unknown): “MFDp [35] http://biomine-ws.ece. ualberta.ca/MFDp.html 3 servers; DISOclust [48], DISOPRED [36], IUPred [39].”
Evaluation of residue-residue contact prediction in CASP10 (2013, background): “As it was discussed in the CASP9 disorder assessment paper 47 , the Matthews correlation coefficient ( MCC ) is a well suited measure for handling cases with imbalanced class frequencies.”
Intrinsically Semi-disordered State and Its Role in Induced Folding and Protein Aggregation (2013, methodology): “Recently, we developed a sequence-based prediction method with integrated neural networks for disorder (SPINE-D) [12] that was ranked as one of the top five performing methods according to area under the curve in the critical assessment of structure prediction techniques (CASP 9) [12, 13].”
Exceptionally abundant exceptions: comprehensive characterization of intrinsic disorder in all domains of life (2014, methodology): “We include such short segments, since they can be predicted with relatively high predictive performance [44] and they were included in some of the similar studies [28, 30].”
High-throughput prediction of RNA, DNA and protein binding regions mediated by intrinsic disorder (2015, background): “Motivated by related works (30,31) we consider the disordered regions with at least 4 consecutive disordered residues.”