Analysis of CASP8 targets, predictions and assessment methods

By Shuoyong Shi, Jimin Pei, Ruslan I. Sadreyev, Lisa N. Kinch, I. Majumdar, Jing Tong, Hua Cheng, Bong-Hyun Kim, Nick V. Grishin. 2009

36 citations recorded by the source corpus.

Results of the recent Critical Assessment of Techniques for Protein Structure Prediction, CASP8, present several valuable sources of information. First, CASP targets comprise a realistic sample of currently solved protein structures and exemplify the corresponding challenges for predictors. Second, the plethora of predictions by all possible methods provides an unusually rich material for evolutionary analysis of target proteins. Third, CASP results show the current state of the field and highlight specific problems in both predicting and assessing. Finally, these data can serve as grounds to develop and analyze methods for assessing prediction quality. Here we present results of our analysis in these areas. Our objective is not to duplicate CASP assessment, but to use our unique experience as former CASP5 assessors and CASP8 predictors to (i) offer more insights into CASP targets and predictions based on expert analysis, including invaluable analysis prior to target structure release; and (ii) develop an assessment methodology tailored towards current challenges in the field. Specifically, we discuss preparing target structures for assessment, parsing protein domains, balancing evaluations based on domains and on whole chains, dividing targets into categories and developing new evaluation scores. We also present evolutionary analysis of the most interesting and challenging targets. Database URL: Our results are available as a comprehensive database of targets and predictions at http://prodata.swmed.edu/CASP8.

Subfields: Materials Chemistry, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

Citations in the corpus

CAD‐score: A new contact area difference‐based function for evaluation of protein structural models (2013, methodology): “11,14–17 However, typically, ‘‘con-tacts’’ in these scores are represented by distances between C a , C b , or all atoms within the arbitrarily specified threshold.”
CASP9 Target Classification (2011, result): “Similar to our analysis of CASP8 targets,6 we found that the average GDT score for models above a calculated random score effectively describes general prediction difficulty.”
Assessment of template based protein structure predictions in CASP9 (2011, methodology): “A discussion of the procedure can be found in Shi et al .26 The target preparation process was carried out using a consensus procedure, with assessors reviewing and confirming each other’s findings until an agreement was reached.”