Assessment of protein structure refinement in CASP9

By Justin L. MacCallum, Alberto Pérez, Michael J. Schnieders, Lan Hua, Matthew P. Jacobson, Ken A. Dill. 2011

97 citations recorded by the source corpus.

We assess performance in the structure refinement category in CASP9. Two years after CASP8, the performance of the best groups has not improved. There are few groups that improve any of our assessment scores with statistical significance. Some predictors, however, are able to consistently improve the physicality of the models. Although we cannot identify any clear bottleneck in improving refinement, several points arise: (1) The refinement portion of CASP has too few targets to make many statistically meaningful conclusions. (2) Predictors are usually very conservative, limiting the possibility of large improvements in models. (3) No group is actually able to correctly rank their five submissions—indicating that potentially better models may be discarded. (4) Different sampling strategies work better for different refinement problems; there is no single strategy that works on all targets. In general, conservative strategies do better, while the greatest improvements come from more adventurous sampling—at the cost of consistency. Comparison with experimental data reveals aspects not captured by comparison to a single structure. In particular, we show that improvement in backbone geometry does not always mean better agreement with experimental data. Finally, we demonstrate that even given the current challenges facing refinement, the refined models are useful for solving the crystallographic phase problem through molecular replacement. Proteins 2011;. © 2011 Wiley‐Liss, Inc.

Subfields: Materials Chemistry, Molecular Biology, Spectroscopy

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Spectroscopy

Citations in the corpus

lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests (2013, methodology): “The optimal value of the inclusion radius parameter R o has been determined on a dataset comprising all CASP9 predictions for multidomain targets, and the corresponding assignment of AUs as defined by the CASP9 assessors.”
3D structures of membrane proteins from genomic sequencing (2012, neutral): “…pair contacts and to identify contacts reﬂecting alternative conformations; (3) the use of fragments imported from known structures; and (4) the use of advanced energy reﬁnement methods, including molecular dynamics and Monte Carlo simulations (Dror et al., 2011; MacCallum et al., 2011).”
GalaxyRefine: protein structure refinement driven by side-chain repacking (2013, methodology): “The GalaxyRefine method has been extensively tested on (i) the refinement category targets of CASP8 (5), CASP9 (6) and CASP10 (53 proteins), (ii) Zhang-server (ITASSER) models (84 proteins) (11) and (iii) ROSETTA server models (69 proteins) (17) for CASP10 templatebased modeling targets and (iv) FG-”
CAD‐score: A new contact area difference‐based function for evaluation of protein structural models (2013, methodology): “In particular, this property of CAD-score may be relevant for assessing model refinement, which turns out to be a surprisingly hard problem.29 Although we developed the new CAD-score with the reference-based model evaluation in mind, the approach may be a valuable tool for other tasks such as…”
Methods in Enzymology: “Flexible backbone sampling methods to model and design protein alternative conformations” (2013, unknown): “Several straightforward global and local metrics have been developed to compare computational predictions of a representative single structure to an experimentally derived X-ray structure (MacCallum et al., 2011).”
Evaluation of predictions in the CASP10 model refinement category (2013, methodology): “…substantial numbers of application-specific integrated circuits, which now allow atomistic MD simulations to run for as long as 100 l s. 17,18 In combination with increasingly accurate physics-based force fields such as CHARMM, which have been used to successfully fold structurally…”
Protein storytelling through physics (2020, background): “CASP’s structure refinement category asks: Are approximately correct structures improvable? Early CASP results showed that trying to improve one SBI algorithm by another SBI method was mostly unsuccessful (91).”
Computational protein structure refinement: Almost there, yet still so far to go (2017, methodology): “64 Beginning with CASP10, MD-based re ﬁ nement became successful as a result of force ﬁ eld improvements, extensive sampling, and the application of ensemble averaging.”
Protein Modelling: What Happened to the “Protein Structure Gap”? (2013, unknown): “Also, comparative models still resemble more closely the template than the target structure, and refinement methods are not able to consistently refine models closer to the target structure (MacCallum et al., 2011).”
Assessment of the model refinement category in CASP12 (2017, background): “1,7,31,32 Refinement targets are selected from amongst the main challenge targets.”