CASP9 Assessment of Free Modeling Target Predictions

By Lisa N. Kinch, Shuoyong Shi, Qian Cong, Hua Cheng, Yuxing Liao, Nick V. Grishin. 2011

101 citations recorded by the source corpus.

We present an overview of the ninth round of Critical Assessment of Protein Structure Prediction (CASP9) “Template free modeling” category (FM). Prediction models were evaluated using a combination of established structural and sequence comparison measures and a novel automated method designed to mimic manual inspection by capturing both global and local structural features. These scores were compared to those assigned manually over a diverse subset of target domains. Scores were combined to compare overall performance of participating groups and to estimate rank significance. Moreover, we discuss a few examples of free modeling targets to highlight the progress and bottlenecks of current prediction methods. Notably, a server prediction model for a single target (T0581) improved significantly over the closest structure template (44% GDT increase). This accomplishment represents the “winner” of the CASP9 FM category. A number of human expert groups submitted slight variations of this model, highlighting a trend for human experts to act as “meta predictors” by correctly selecting among models produced by the top‐performing automated servers. The details of evaluation are available at http://prodata.swmed.edu/CASP9/. © Proteins 2011; © 2011 Wiley‐Liss, Inc.

Subfields: Materials Chemistry, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

Citations in the corpus

Protein structure prediction from sequence variation (2012, background): “…de novo predictions from sequence, when not a single structure in a protein family is known, have been hard to achieve, as the pioneering Critical Assessment of Techniques for Protein Structure Prediction (CASP) evaluation of blinded predictions has demonstrated over the past two decades 5,6 .”
Assessment of hard target modeling in CASP12 reveals an emerging role of alignment‐based contact prediction methods (2018, methodology): “Values above 50 generally indicate overall good topology.(4) However, as noted in previous CASPs, models ranked top by GDTTS might not always match the best ones identified by visual examination, thus it is important to use additional metrics to guide visual expert assessment.”
Assessment of CASP10 contact‐assisted predictions (2014, methodology): “We chose two scores, GDT-TS,10 often abbreviated here as GDT, and QCS,11,12 created for CASP9 FM assessment, to rank the predictions.”
Folding non-homologous proteins by coupling deep-learning contact maps with I-TASSER assembly simulations (2021, background): “Early efforts in using contact maps to fold proteins, however, enjoyed modest to little success (Kinch et al., 2011; Wu et al., 2011), mainly due to the low accuracy of contact-map predictions, which had typically less than 30% of the top L/5 long-range contacts, where L is the protein length and lo”
Assessment of template-free modeling in CASP10 and ROLL (2013, methodology): “Purely score-based, automated ranking of predictions works well for TBM targets, but it is generally acknowledged that manual expert evaluation is required to properly assess FM predictions1–3 and, as was done by previous FM assessors, we also evaluated these models ultimately by visual inspection.”
GalaxyRefine: protein structure refinement driven by side-chain repacking (2013, background): “Although ab initio protein structure predictions from sequences are notoriously difficult (3,4), ab initio refinement starting from a reasonable initial model structure is expected to be less difficult.”
Automatic Prediction of Protein 3D Structures by Probabilistic Multi-template Homology Modeling (2015, methodology): “Homology modeling is by far the most widely used computational approach to predict the 3D structures of proteins, and almost all protein structure prediction servers rely chiefly on homology modeling, as seen in the community-wide blind benchmark “Critical Assessment of Techniques for Protein Struct”
Template-based and free modeling of I-TASSER and QUARK pipelines using predicted contact maps in CASP12 (2017, background): “5,13,14,17,18 One of the primary reasons for the difficulty of ab initio modeling in predicting large proteins with complicated topologies is the lack of precise long-range contact interaction information in the force field during protein structure assembly simulation.”
Chemical shift-based methods in NMR structure determination (2018, methodology): “However, modeling structures exclusively from their sequence is extremely challenging and limited in scope to smaller systems as highlighted by the CASP (Critical Assessment of Techniques for Protein Structure Prediction) initiative [21,22].”