Learning inverse folding from millions of predicted structures

By Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian L. Hie, Tom Sercu, Adam Lerer, Alexander Rives. 2022

618 citations recorded by the source corpus.

We consider the problem of predicting a protein sequence from its backbone atom coordinates. Machine learning approaches to this problem to date have been limited by the number of available experimentally determined protein structures. We augment training data by nearly three orders of magnitude by predicting structures for 12M protein sequences using AlphaFold2. Trained with this additional data, a sequence-to-sequence transformer with invariant geometric input processing layers achieves 51% native sequence recovery on structurally held-out backbones with 72% recovery for buried residues, an overall improvement of almost 10 percentage points over existing methods. The model generalizes to a variety of more complex tasks including design of protein complexes, partially masked structures, binding interfaces, and multiple states.

Subfields: Materials Chemistry, Molecular Biology, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Computational Biology & Bioinformatics

Citations in the corpus

Robust deep learning based protein sequence design using ProteinMPNN (2022, positive): “We first sought to improve performance of the model on recovering the amino acid sequences of native monomeric proteins given their backbone structures, using as training and validation sets 19.7k high resolution single-chain structures from the PDB split based on the CATH (7) protein classification”
Evolutionary-scale prediction of atomic level protein structure with a language model (2022, background): “[62] Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives.”
Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models (2022, background): “Similarly, machine learning approaches have come to perform well for the sequence generation inverse problem [3, 21, 20].”
Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem (2022, neutral): “Another class of methods, referred to as fixed backbone sequence design [12, 20, 40, 27, 18], attempts to solve the problem of identifying a sequence that folds into any given designable backbone structure.”
Masked Inverse Folding with Sequence Transfer for Protein Representation Learning (2023, background): “…folding is usually parametrized as a graph neural network (GNN) (Ingra-ham et al., 2019; Strokach et al., 2020; Jin et al., 2021; Jing et al., 2020; Hsu et al., 2022; Dauparas et al., 2022; Shi et al., 2022) or SE(3)-equivariant transformer (McPartlon et al., 2022) that either reconstructs or…”
Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures (2022, background): “Fix-backbone design is a common setting in the area of protein design [21, 20, 6, 46, 47].”
Scaling Laws for Generative Mixed-Modal Language Models (2023, background): “…language models have been developed for a wide range of data modalities, including nat-ural language text Brown et al. (2020), code (Chen et al., 2021; Fried et al., 2022), images (Ramesh et al., 2021; Yasunaga et al., 2022), and molecules or proteins (Chilingaryan et al., 2022; Hsu et al., 2022).”
Efficient and accurate prediction of protein structure using RoseTTAFold2 (2023, methodology): “Recycling – running multiple iterations of the network for each prediction – and using AF2 predictions 10 to augment the training set, improved the GDT-TS by 3.1.”
Scientific discovery in the age of artificial intelligence (2023, background): “Such representations are trans-ferable across various tasks, ranging from sequence design 103–105 to structure prediction 10,106 .”
SaProt: Protein Language Modeling with Structure-aware Vocabulary (2024, methodology): “SaProt consistently exhibited improved performance compared to strong baselines, particularly models from the ESM family, including ESM-1b, ESM-1v (Meier et al., 2021), and ESM-2 (Lin et al., 2022), which are considered leading PLMs in the field.”
ProGen2: Exploring the Boundaries of Protein Language Models (2022, methodology): “Motivated by this work, ESM-IF1 (Hsu et al., 2022) replaced the encoder network with an rotation-equivariant model (Jing et al.”
Illuminating protein space with a programmable generative model (2022, neutral): “23–33 and Supplementary Appendices N–”
Protein structure generation via folding diffusion (2022, methodology): “Following this general procedure, for a generated structure s, we use the ESM-IF1 inverse folding model (Hsu et al., 2022) to generate 8 different amino acid sequences.”