Single-sequence protein structure prediction using language models and deep learning

By Ratul Chowdhury, Nazim Bouatta, Surojit Biswas, C. Floristean, Anant Kharkare, Koushik Roye, Charlotte Rochereau, Gustaf Ahdritz, Joanna Zhang, George M. Church and 2 more. 2022

358 citations recorded by the source corpus.

AlphaFold2 and related computational systems predict protein structure using deep learning and co-evolutionary relationships encoded in multiple sequence alignments (MSAs). Despite high prediction accuracy achieved by these systems, challenges remain in (1) prediction of orphan and rapidly evolving proteins for which an MSA cannot be generated; (2) rapid exploration of designed structures; and (3) understanding the rules governing spontaneous polypeptide folding in solution. Here we report development of an end-to-end differentiable recurrent geometric network (RGN) that uses a protein language model (AminoBERT) to learn latent structural information from unaligned proteins. A linked geometric module compactly represents Cα backbone geometry in a translationally and rotationally invariant way. On average, RGN2 outperforms AlphaFold2 and RoseTTAFold on orphan proteins and classes of designed proteins while achieving up to a 106-fold reduction in compute time. These findings demonstrate the practical and theoretical strengths of protein language models relative to MSAs in structure prediction. RGN2 predicts a protein’s structure from its sequence without a multiple sequence alignment.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

AlphaFold2 and its applications in the fields of biology and medicine (2023, methodology): “…modeling methods based on DL, among which AlphaFold, 1,47 RoseTTAFold, 48 ESMFold 49 (ESMFold also offers an extensive database of protein structural predictions, which include 617 million metagenomic protein structures) and the recent language model by Chowdhury et al. 50 are the most famous ones.”
State-specific protein–ligand complex structure prediction with a multiscale deep generative model (2022, positive): “Specifically, evolutionary constraints extracted from multiple sequence alignments (MSAs) 2–5 or protein language models (PLMs) 6–9 and specialized neural networks are systematically combined with the sequence-based Article https://doi.org/10.1038/s42256-024-00792-z the physics-based methods…”
Clustering predicted structures at the scale of the known protein universe (2023, methodology): “Improvements in methods for predicting structure from sequences 1,3,4 now enable the scalable prediction of protein structures for the known protein universe.”
Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies (2022, methodology): “Embeddings from transformer encoder models trained for masked language modeling have been used for variant prediction 20 , evolutionary analysis 21,22 , and as features for protein structure prediction 23,24 .”
Protein structure generation via folding diffusion (2022, methodology): “While a similar angular representation has been used in some protein structure prediction works 22 – 24 , it has only received cursory exploration in the context of generative modeling, and only for simple, helix-only protein structures 25 .”
Biasing AlphaFold2 to predict GPCRs and kinases with user-defined functional or structural properties (2022, neutral): “Since the emergence of AF2, a number of deep learning-based methods have been developed with the same goal of predicting protein structures at experimental accuracy (AlQuraishi, 2021; Baek et al., 2021; Chowdhury et al., 2022; Lin et al., 2022).”
OpenFold: retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization (2023, neutral): “It is possible that, in earlier phases of training, OpenFold has already recovered aspects of secondary structure not recognized by DSSP on account of imprecise atom positioning.”