Estimation of model accuracy in CASP13

By Jianlin Cheng, Myong-Ho Choe, Arne Elofsson, Kun-Sop Han, Jie Hou, Ali Hassan Ahmed Maghrabi, Liam J. McGuffin, David Menéndez-Hurtado, Kliment Olechnovic, Torsten Schwede and 4 more. 2019

75 citations recorded by the source corpus.

Methods to reliably estimate the accuracy of 3D models of proteins are both a fundamental part of most protein folding pipelines and important for reliable identification of the best models when multiple pipelines are used. Here, we describe the progress made from CASP12 to CASP13 in the field of estimation of model accuracy (EMA) as seen from the progress of the most successful methods in CASP13. We show small but clear progress, that is, several methods perform better than the best methods from CASP12 when tested on CASP13 EMA targets. Some progress is driven by applying deep learning and residue‐residue contacts to model accuracy prediction. We show that the best EMA methods select better models than the best servers in CASP13, but that there exists a great potential to improve this further. Also, according to the evaluation criteria based on local similarities, such as lDDT and CAD, it is now clear that single model accuracy methods perform relatively better than consensus‐based methods.

Subfields: Materials Chemistry, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

Citations in the corpus

Introducing “best single template” models as reference baseline for the Continuous Automated Model Evaluation (CAMEO) (2019, methodology): “A very important way to communicate a model's quality to the end user is by assigning correct per-residue confidence estimates.(27) “SWISS-MODEL,” “IntFOLD3-TS,” and “Robetta” are providing good to very good error estimates with “SWISS-MODEL” and “IntFOLD3-TS” peaking at 0.”
Learning from Protein Structure with Geometric Vector Perceptrons (2020, methodology): “SASHAN learns a linear model over secondary structure and contact-based features (Cheng et al., 2019).”
GraphQA: protein model quality assessment using graph convolutional networks (2020, unknown): “…artificial neural networks (Wallner and Elofsson, 2006) or support vector machines (Ray et al. , 2012; Uziela et al. , 2016) to more recent works including MULTICOM (Hou et al. , 2019), SARTclust (submitted as group UOSHAN in CASP13) (Cheng et al. , 2019), ModFOLD7 (McGuffin et al. , 2019b),…”
Integrating model simulation tools and cryo‐electron microscopy (2022, methodology): “241 These methods have been used successfully to score predicted models in the CASP competitions, 242 but are also applicable to models refined into cryo-EM density.”
Assessment of Protein Model Structure Accuracy Estimation in CASP13: Challenges in the Era of Deep Learning (2019, methodology): “pated in CASP12, when the previous versions were tested on the same CASP13 EMA targets.(20)”