Assessment of the model refinement category in CASP12

By Ladislav Hovan, Vladimiras Oleinikovas, Havva Yalinca, Andriy Kryshtafovych, Giorgio Saladino, Francesco Luigi Gervasio. 2017

35 citations recorded by the source corpus.

We here report on the assessment of the model refinement predictions submitted to the 12th Experiment on the Critical Assessment of Protein Structure Prediction (CASP12). This is the fifth refinement experiment since CASP8 (2008) and, as with the previous experiments, the predictors were invited to refine selected server models received in the regular (nonrefinement) stage of the CASP experiment. We assessed the submitted models using a combination of standard CASP measures. The coefficients for the linear combination of Z‐scores (the CASP12 score) have been obtained by a machine learning algorithm trained on the results of visual inspection. We identified eight groups that improve both the backbone conformation and the side chain positioning for the majority of targets. Albeit the top methods adopted distinctively different approaches, their overall performance was almost indistinguishable, with each of them excelling in different scores or target subsets. What is more, there were a few novel approaches that, while doing worse than average in most cases, provided the best refinements for a few targets, showing significant latitude for further innovation in the field.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Molecular Biology

Citations in the corpus

Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XII (2017, methodology): “In CASP12, the accuracy of the best single-model methods (black line) is higher than that of clustering methods (black dashed line), while in CASP11 the accuracy of single-model meth-ods (red solid line) was much worse than the accuracy of clustering methods (red dashed line) CASP9, assessors have i”
High-accuracy refinement using Rosetta in CASP13 (2019, neutral): “the actual structure, the approach more often reduced rather than increased the accuracy of the starting models.(2,3) We reasoned that”
GalaxyRefine2: simultaneous refinement of inaccurate local regions and overall protein structure (2019, methodology): “The GalaxyRefine2 server was tested in the refinement category of CASP12 and CASP13 in a blinded manner, and the server named ‘Seok-server’ was ranked highly among all servers (18,29).”
Protein homology model refinement by large-scale energy optimization (2018, methodology): “…simpler and quantitative, we use a single metric — SphereGrinder (SG) (23) — which best captures large-scale structural improvements (3, 8, 11, 22); over the benchmark set the SG values are correlated with both the rmsd and the GDT-HA (global distance test-high accuracy) (24) ( SI Appendix ,…”
Evaluation of model refinement in CASP13 (2019, methodology): “In CASP8,(1) only one group (Lee) succeeded in improving the average global distance test total score (GDT_TS)(7) from the starting models, whereas by the time of CASP12 8 of 39 groups succeeded in improving the more stringent high-accuracy GDT_HA score.(5)”