Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XII

By John Moult, Krzysztof Fidelis, Andriy Kryshtafovych, Torsten Schwede, Anna Tramontano. 2017

371 citations recorded by the source corpus.

This article reports the outcome of the 12th round of Critical Assessment of Structure Prediction (CASP12), held in 2016. CASP is a community experiment to determine the state of the art in modeling protein structure from amino acid sequence. Participants are provided sequence information and in turn provide protein structure models and related information. Analysis of the submitted structures by independent assessors provides a comprehensive picture of the capabilities of current methods, and allows progress to be identified. This was again an exciting round of CASP, with significant advances in 4 areas: (i) The use of new methods for predicting three‐dimensional contacts led to a two‐fold improvement in contact accuracy. (ii) As a consequence, model accuracy for proteins where no template was available improved dramatically. (iii) Models based on a structural template showed overall improvement in accuracy. (iv) Methods for estimating the accuracy of a model continued to improve. CASP continued to develop new areas: (i) Assessing methods for building quaternary structure models, including an expansion of the collaboration between CASP and CAPRI. (ii) Modeling with the aid of experimental data was extended to include SAXS data, as well as again using chemical cross‐linking information. (iii) A team of assessors evaluated the suitability of models for a range of applications, including mutation interpretation, analysis of ligand binding properties, and identification of interfaces. This article describes the experiment and summarizes the results. The rest of this special issue of PROTEINS contains papers describing CASP12 results and assessments in more detail.

Subfields: Materials Chemistry, Molecular Biology, Evaluation & Benchmarks, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Evaluation & Benchmarks

In Computational Biology & Bioinformatics

Citations in the corpus

Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences (2019, neutral): “The Test (59) and CASP11 (60) test sets evaluate with sequence identity holdout at 25%, the CASP12 (61) test set implements a temporal holdout with the structural training data, and the CASP13 (58) experiment implements a full temporal holdout of both the pretraining and training data.”
Improved protein structure refinement guided by deep learning based accuracy estimation (2020, neutral): “However, in most cases the predicted structures still deviate considerably from the actual structure (4).”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, positive): “We used two protein sequence datasets from the TAPE repository for the analysis: the ProteinNet dataset [3, 10, 26, 52] and the Secondary Structure dataset [10, 38, 52, 60].”
Evaluating Protein Transfer Learning with TAPE (2019, background): “The most well-known protein modeling benchmark is the Critical Assessment of Structure Prediction (CASP) [26], which focuses on structure modeling.”
DEEPCON: Protein Contact Prediction using Dilated Convolutional Neural Networks with Dropout (2019, methodology): “Following the standard in the field [4], as evaluation metrics of contact prediction accuracy, we use the precision of top L/5 predicted long-range contacts (PTOP-L/5-LR), where L is the length of the protein sequence.”
End-to-end differentiable learning of protein structure (2018, background): “More substantial gains were seen in CASP 12 due to the use of co‐evolutionary information (Moult John et al., 2018), but RGNs match these advances without using co‐evolutionary data and by operating in a fundamentally distinct and complementary way.”
ProteinNet: a standardized data set for machine learning of protein structure (2019, methodology): “With respect to standardized training / validation / test splits, the closest existing analogues are the biennial Critical Assessment of protein Structure Prediction (CASP) (Moult John et al. , 2018) and the continually running Continuous Automated Model EvaluatiOn (CAMEO) (Haas et al. , 2018).”
Protein model quality assessment using 3D oriented convolutional neural networks (2018, background): “This problem is challenging as it is shown by the fact that the Critical Assessment of protein Structure Prediction (CASP) community experiment (1) has an entire category dedicated to this specific topic (2).”
Using Rosetta for RNA homology modeling (2019, methodology): “For illustration, in the allied field of protein structural modeling, modeling based on templates has yielded substantially more accurate models than de novo modeling, even for proteins with limited similarity to existing structures (Moult, Fidelis, Kryshtafovych, Schwede, & Tramontano, 2018).”
Protein language model embeddings for fast, accurate, alignment-free protein structure prediction (2021, methodology): “Panel B: SetValCASP12 + SetTstCASP13: the five violin plots for embeddings from four different pLMs (Elnaggar et al., 2021; Heinzinger et al., 2019) along with our in-house method using evolutionary couplings (DCAdst).”
Assessing the accuracy of contact predictions in CASP13 (2019, background): “started in 1994 and have continued biannually ever since, with the latest meeting, CASP13, held in 2018.(1) In advance of the meeting, the participating groups make blind predictions for a number of protein structures, complexes, or contacts, and these predictions are assessed”