Assessment of Contact Predictions in CASP12: Co-Evolution and Deep Learning Coming of Age

By J. Schaarschmidt, Bohdan Monastyrskyy, Andriy Kryshtafovych, Alexandre M. J. J. Bonvin. 2017

180 citations recorded by the source corpus.

Following up on the encouraging results of residue‐residue contact prediction in the CASP11 experiment, we present the analysis of predictions submitted for CASP12. The submissions include predictions of 34 groups for 38 domains classified as free modeling targets which are not accessible to homology‐based modeling due to a lack of structural templates. CASP11 saw a rise of coevolution‐based methods outperforming other approaches. The improvement of these methods coupled to machine learning and sequence database growth are most likely the main driver for a significant improvement in average precision from 27% in CASP11 to 47% in CASP12. In more than half of the targets, especially those with many homologous sequences accessible, precisions above 90% were achieved with the best predictors reaching a precision of 100% in some cases. We furthermore tested the impact of using these contacts as restraints in ab initio modeling of 14 single‐domain free modeling targets using Rosetta. Adding contacts to the Rosetta calculations resulted in improvements of up to 26% in GDT_TS within the top five structures.

Subfields: Materials Chemistry, Molecular Biology, Representation & Self-Supervised Learning, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Representation & Self-Supervised Learning

In Computational Biology & Bioinformatics

Citations in the corpus

Evaluating Protein Transfer Learning with TAPE (2019, background): “This increased specificity is highly relevant in the structure prediction pipeline [39, 52] and highlights a clear challenge for pretraining.”
Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XII (2017, methodology): “Assessment of model accuracy estimates has been included in CASP since CASP7 (2006) 16 and steady improvement in the accuracy of the methods was reported in CASPs 8, 9, 10, and 11.”
Enhancing Evolutionary Couplings with Deep Convolutional Neural Networks (2017, methodology): “We participated in the CASP12 experiment finishing with the 2 best average rank across the variety of categories and ranking metrics considered, as well as in the top two methods based on average F1 score, with the other top method, RaptorX-Contact, being another CNN-based model using a much deeper ”
Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV (2021, background): “But in 2016, the accuracy from the best groups almost doubled to just under 50%.(33) Apparently, this was a consequence of the introduction of improved classical statistical methods.”
A further leap of improvement in tertiary structure prediction in CASP13 prompts new routes for future assessments (2019, background): “In CASP12, the main improvements in modeling of hard targets were brought about by the initial prediction of residue-residue contacts, in which an independent assessment showed had improved substantially from CASP11.”
Driven to Near-Experimental Accuracy by Refinement via Molecular Dynamics Simulations (2019, background): “3 7-11 If sufficient residue contact information is available, high-resolution models can be built even when there is no homologous structure available as a template.”
ComplexContact: a web server for inter-protein contact prediction using deep learning (2018, methodology): “The underlying algorithm of this server is a Deep Learning (DL) model which has won intra-protein contact prediction in CASP12 (24,25).”
Coevolutionary Analysis of Protein Subfamilies by Sequence Reweighting (2019, methodology): “This can be seen in the high scores obtained by the top-ranking teams in the recent CASP competition (Critical Assessment of Protein Structure Prediction), which mostly rely on coevolutionary predictions to guide their structural modelling [6].”
CASP13 Target Classification into Tertiary Structure Prediction Categories (2019, methodology): “contributed to model accuracy of top performing non-template-based methods.(1,37,38) Table 1 summarizes the evolutionary relationship of CASP13 targets to known folds that guided classification.”