Assessment of hard target modeling in CASP12 reveals an emerging role of alignment‐based contact prediction methods

By Luciano A. Abriata, Giorgio Tamò, Bohdan Monastyrskyy, Andriy Kryshtafovych, M. dal Peraro. 2018

90 citations recorded by the source corpus.

We present our assessment of CASP12 modeling efforts for targets with no obvious templates of high sequence/structure similarity in the PDB, that is for evaluation units of the free modeling (FM) and free modeling/template‐based modeling (FM/TBM) categories. Models were clustered and ranked using the Global Distance Test‐Total Score and 5 additional metrics developed in previous CASP rounds, producing short lists of models that were subject to visual inspection in comparison to the target structures. The whole procedure was implemented as a web app that facilitates model selection and visual inspection, and could become useful to facilitate and standardize future assessments. We describe cases of (1) targets with remarkably good predictions, (2) targets whose models captured some global shape and topology features, and (3) targets for which models fail to capture even coarse features. We note that despite this CASP being among the most challenging ones, a measurable improvement of the top predictions is apparent, that we attribute to the emergence of accurate contact prediction methods and the increased number of available sequences. We also briefly discuss current limitations in tertiary structure prediction exemplified by CASP12 targets. Overall, the Baker, Zhang, and Lee manual groups and servers were identified as the top global performing groups.

Subfields: Materials Chemistry, Molecular Biology, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Computational Biology & Bioinformatics

Citations in the corpus

Modeling aspects of the language of life through transfer-learning protein sequences (2019, neutral): “…protein of known structure in the PDB at the time); CB513 [70]: 513 non-redundant sequences compiled 20 years ago (511 after SIFTS mapping); CASP12 [71]: 21 proteins taken from the CASP12 free-modelling targets (20 after SIFTS mapping; all 21 fulfilled a stricter criterion toward non-redundancy…”
NetSurfP-2.0: improved prediction of protein structural features by integrated deep learning (2018, background): “2016), the CB513 datasets (513 protein regions from 434 proteins) (31), and a dataset consisting of all the free-modeling targets (21 proteins) at the CASP 12 experiment (32).”
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing (2020, methodology): “The final performance was evaluated on three different data sets, each with <25% PIDE to the training set: CB513 (513 proteins; [41]), TS115 (115 proteins; [42]) and CASP12 (21 proteins; [43]).”
Deep-learning contact-map guided protein structure prediction in CASP13 (2019, methodology): “7,8 In previous Critical Assessment of Protein Structure Prediction (CASP) experiments, we developed and tested two pipelines, “ Zhang-Server ” and “ QUARK ” , in the TBM and FM categories, respectively.”
Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization (2020, background): “Given the diversity of biological use cases, embedding evaluations should be made on a case-by-case basis.”
NetSurfP-3.0: accurate and fast prediction of protein structural features by protein language models and deep learning (2022, methodology): “Its training and validation set consist of 10 337 and 500 non-redundant proteins, respectively, and the testing is performed on the three test sets TS115 (115 proteins) (17), CB513 (513 protein regions from 434 proteins) (18), and CASP12 (21 proteins) (19).”
Ensembling multiple raw coevolutionary features with deep residual neural networks for contact-map prediction in CASP13 (2019, background): “Significant progress has been recently witnessed on ab initio 3D structure prediction,(4-7) which is mainly due to the success of sequence-based contact predictions.(8,9) Due to its significant importance to protein structure prediction, the problem of contact-map prediction has drawn an increasing ”
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Learning (2021, unknown): “We also included other public test data sets, namely CB513 [47]), TS115 [48], and CASP12 [49].”
Bilingual language model for protein sequence and structure (2024, unknown): “We benchmarked using three public test data sets, namely CASP12 [38], CASP14 and NEW364 [4].”
Topology Evaluation of Models for Difficult Targets in the 14th Round of the Critical Assessment of Protein Structure Prediction (CASP14) (2021, background): “4,5,11 The ranking formula also included an aggregated score (MolProbity) of prediction model quality that combined the clash, rotamer outlier, and Ramachandran favored components of experimental structure model evaluation.”