Assessment of Protein Model Structure Accuracy Estimation in CASP13: Challenges in the Era of Deep Learning

By Jonghun Won, Minkyung Baek, Bohdan Monastyrskyy, Andriy Kryshtafovych, Chaok Seok. 2019

56 citations recorded by the source corpus.

Scoring model structure is an essential component of protein structure prediction that can affect the prediction accuracy tremendously. Users of protein structure prediction results also need to score models to select the best models for their application studies. In Critical Assessment of techniques for protein Structure Prediction (CASP), model accuracy estimation methods have been tested in a blind fashion by providing models submitted by the tertiary structure prediction servers for scoring. In CASP13, model accuracy estimation results were evaluated in terms of both global and local structure accuracy. Global structure accuracy estimation was evaluated by the quality of the models selected by the global structure scores and by the absolute estimates of the global scores. Residue‐wise, local structure accuracy estimations were evaluated by three different measures. A new measure introduced in CASP13 evaluates the ability to predict inaccurately modeled regions that may be improved by refinement. An intensive comparative analysis on CASP13 and the previous CASPs revealed that the tertiary structure models generated by the CASP13 servers show very distinct features. Higher consensus toward models of higher global accuracy appeared even for free modeling targets, and many models of high global accuracy were not well optimized at the atomic level. This is related to the new technology in CASP13, deep learning for tertiary contact prediction. The tertiary model structures generated by deep learning pose a new challenge for EMA (estimation of model accuracy) method developers. Model accuracy estimation itself is also an area where deep learning can potentially have an impact, although current EMA methods have not fully explored that direction.

Subfields: Materials Chemistry, Molecular Biology, Spectroscopy

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Spectroscopy

Citations in the corpus

Learning from Protein Structure with Geometric Vector Perceptrons (2020, background): “Thanks to their importance and difficulty, such problems, which we broadly refer to as learning from structure, have recently developed into an exciting and promising application area for deep learning (Graves et al., 2020; Ingraham et al., 2019; Pereira et al., 2016; Townshend et al., 2019; Won et ”
Ins and outs of AlphaFold2 transmembrane protein structure predictions (2022, unknown): “There was a change at the 13th Critical Assessment of Protein Structure Prediction (CASP13) competition [2] when the neural network-based approach, AlphaFold excelled.”
Application of docking methodologies to modeled proteins (2020, background): “These tools evolve rapidly, 49 and an assessment based on them would be soon out of date.”
GraphQA: protein model quality assessment using graph convolutional networks (2020, unknown): “Single-model methods are applicable to a single protein in isolation and in the recent CASP13 performed comparably to or better than consensus methods for the first time (Cheng et al. , 2019).”