Evaluation of residue-residue contact prediction in CASP10

By Bohdan Monastyrskyy, Daniel D'Andrea, Krzysztof Fidelis, Anna Tramontano, Andriy Kryshtafovych. 2013

85 citations recorded by the source corpus.

We present the results of the assessment of the intramolecular residue‐residue contact predictions from 26 prediction groups participating in the 10th round of the CASP experiment. The most recently developed direct coupling analysis methods did not take part in the experiment likely because they require a very deep sequence alignment not available for any of the 114 CASP10 targets. The performance of contact prediction methods was evaluated with the measures used in previous CASPs (i.e., prediction accuracy and the difference between the distribution of the predicted contacts and that of all pairs of residues in the target protein), as well as new measures, such as the Matthews correlation coefficient, the area under the precision‐recall curve and the ranks of the first correctly and incorrectly predicted contact. We also evaluated the ability to detect interdomain contacts and tested whether the difficulty of predicting contacts depends upon the protein length and the depth of the family sequence alignment. The analyses were carried out on the target domains for which structural homologs did not exist or were difficult to identify. The evaluation was performed for all types of contacts (short, medium, and long‐range), with emphasis placed on long‐range contacts, i.e. those involving residues separated by at least 24 residues along the sequence. The assessment suggests that the best CASP10 contact prediction methods perform at approximately the same level, and comparably to those participating in CASP9. Proteins 2014; 82(Suppl 2):138–153. © 2013 Wiley Periodicals, Inc.

Subfields: Materials Chemistry, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

Citations in the corpus

Analysis of several key factors influencing deep learning-based inter-residue contact prediction (2019, background): “An approximate 3D protein structure can be built with good accuracy if a sufficient amount of accurately predicted residue–residue contacts are available (Marks et al., 2011; Monastyrskyy et al., 2014).”
CONFOLD: Residue-Residue Contact-guided ab initio Protein Folding (2015, methodology): “We used two standard protein data sets for our experiments: (1) 15 test proteins of different fold classes ranging from 48 to 248 residues used in EVFOLD,9 and (2) 150 diverse globular proteins with average length of 145 residues used in FRAGFOLD.”
Folding non-homologous proteins by coupling deep-learning contact maps with I-TASSER assembly simulations (2021, background): “, 2011),mainly due to the lowaccuracy of contact-map predictions, which had typically less than 30% of the top L/5 long-range contacts, where L is the protein length and long range represents a sequence separation with at least 24 residues, correctly predicted (Monastyrskyy et al., 2014).”
De novo Protein Structure Prediction by Coupling Contact With Distance Profile (2020, background): “Currently, researchers consider that the top L , L= 2 , L= 5 and L= 10 are reasonable, where L is the length of the protein sequence, and the rank of contacts is determined by the prediction conﬁdence score [15], [16], [48].”
Supporting Information S1: Metrics of contact correctness (2014, unknown): “This kind of contact evaluation criterion has been used for some time in the biannual Critical Assessment of protein Structure Prediction (CASP) competition, and is considered standard in the field of protein structure prediction [5, 6].”
CASP PREDICTION CENTER INFRASTRUCTURE AND EVALUATION MEASURES IN CASP10 AND CASP ROLL (2013, background): “13,14 In the DR category we also begun to require that prediction of a residue in a disordered state should be necessarily accompanied by a probability value in the range 0.5–1.0.”
ConEVA: a toolbox for comprehensive assessment of protein contacts (2016, methodology): “In addition, other measures like ‘ mean false positive error ’ , ‘ distance in contact map ’ or ‘ spread ’ [13], F-score and Matthews correlation coefficient (MCC) [12] are also used for a more rigorous evaluation of the predicted contacts.”