Single-sequence protein structure prediction using language models from deep learning
By Ratul Chowdhury, Nazim Bouatta, Surojit Biswas, Charlotte Rochereau, George M. Church, Peter K. Sorger, Mohammed Alquraishi. 2021
64 citations recorded by the source corpus.
AlphaFold2 and related systems use deep learning to predict protein structure from co-evolutionary relationships encoded in multiple sequence alignments (MSAs). Despite dramatic, recent increases in accuracy, three challenges remain: (i) prediction of orphan and rapidly evolving proteins for which an MSA cannot be generated, (ii) rapid exploration of designed structures, and (iii) understanding the rules governing spontaneous polypeptide folding in solution. Here we report development of an end-to-end differentiable recurrent geometric network (RGN) able to predict protein structure from single protein sequences without use of MSAs. This deep learning system has two novel elements: a protein language model (AminoBERT) that uses a Transformer to learn latent structural information from millions of unaligned proteins and a geometric module that compactly represents Cα backbone geometry. RGN2 outperforms AlphaFold2 and RoseTTAFold (as well as trRosetta) on orphan proteins and is competitive with designed sequences, while achieving up to a 106-fold reduction in compute time. These findings demonstrate the practical and theoretical strengths of protein language models relative to MSAs in structure prediction.
Subfields: Molecular Biology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Molecular Biology
- Jeffrey J. Gray
- Sergey Ovchinnikov
- David Baker
- Arne Elofsson
- Ivan V. Anishchenko
- Kevin Kaichuang Yang
- Sai Pooja Mahajan
- Jeffrey A. Ruffolo
Citations in the corpus
- Improved prediction of protein-protein interactions using AlphaFold2 (2021, methodology): “Since then, other end-to-end structure predictors have emerged using different principles such as fast MSA processing in DMPFold2 17 and language model representations 18 .”
- Masked Inverse Folding with Sequence Transfer for Protein Representation Learning (2023, background): “Pretrained protein MLMs contain structural information (Rao et al., 2019; Rives et al., 2021; Chowdhury et al., 2021), encode evolutionary trajectories (Hie et al., 2022b; 2021), are zero-shot predictors of mutation fitness effects (Meier et al., 2021), improve out-of-domain generalization on…”
- Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies (2022, methodology): “Embeddings from transformer encoder 28 models trained for masked language modeling have been used for variant prediction (19), evolutionary analysis (20, 21), 29 and as features for protein structure prediction (22, 23).”
- Deep learning methods for designing proteins scaffolding functional sites (2021, methodology): “It is impractical to experimentally validate many designs for many different applications; we instead evaluate these designs using the AlphaFold (AF) protein structure prediction network ( 16 ) which has very high accuracy on de novo designed proteins ( 17 ).”