Protein design and variant prediction using autoregressive generative models

By Jung-Eun Shin, Adam J. Riesselman, Aaron W. Kollasch, Conor McMahon, Elana Simon, Christopher Sander, A. Manglik, A. Kruse, Debora Marks. 2021

363 citations recorded by the source corpus.

The ability to design functional sequences and predict effects of variation is central to protein engineering and biotherapeutics. State-of-art computational methods rely on models that leverage evolutionary information but are inadequate for important applications where multiple sequence alignments are not robust. Such applications include the prediction of variant effects of indels, disordered proteins, and the design of proteins such as antibodies due to the highly variable complementarity determining regions. We introduce a deep generative model adapted from natural language processing for prediction and design of diverse functional sequences without the need for alignments. The model performs state-of-art prediction of missense and indel effects and we successfully design and test a diverse 105-nanobody library that shows better expression than a 1000-fold larger synthetic library. Our results demonstrate the power of the alignment-free autoregressive model in generalizing to regions of sequence space traditionally considered beyond the reach of prediction and design. The ability to design functional sequences is central to protein engineering and biotherapeutics. Here the authors introduce a deep generative alignment-free model for sequence design applied to highly variable regions and design and test a diverse nanobody library with improved properties for selection experiments.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

Learning inverse folding from millions of predicted structures (2022, background): “…Eguchi et al., 2020), jointly generate structures and sequences (An-ishchenko et al., 2021; Wang et al., 2021), or model sequences directly (Rives et al., 2021; Madani et al., 2021; Shin et al., 2021; Gligorijevic et al., 2021; Bryant et al., Augmenting inverse folding with predicted structures.”
Using AlphaFold for Rapid and Accurate Fixed Backbone Protein Design (2021, background): “…variational autoencoders (Greener et al., 2018; Hawkins-Hooker et al., 2021), deep exploration networks (Linder et al., 2020), graph neural networks (Strokach et al., 2020), recurrent neural networks (Alley et al., 2019) and autoregressive models (Shin et al., 2021; Trinquier et al., 2021).”
Function-guided protein design by deep manifold sampling (2021, methodology): “Additionally, a few generative models have been proposed for conditional design [17, 6, 8]: Greener et al. [17] use a VAE conditioned on structural features for generating sequences with metal binding sites, Madani et al. [6] use a conditional LM for sampling proteins, where each amino acid is…”
ProGen2: Exploring the Boundaries of Protein Language Models (2022, methodology): “More versatile models trained on unaligned and unrelated sequences have emerged (Shin et al., 2021) for functional sequence design.”
FLIP: Benchmark tasks in fitness landscape inference for proteins (2021, methodology): “Unsupervised models were also applied to-or built on evolutionary sequence inputs to model the effects of mutations [18–21].”
Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies (2022, background): “Generative models trained on large numbers of natural antibody sequences 341 can produce e ective libraries for antibody discovery (28, 29).”
Protein structure generation via folding diffusion (2022, methodology): “Autoregressive training strategies have been successfully applied to generative modeling of sequences of language tokens in natural language processing 42,43 and amino acid tokens in protein language models 44 – 46 .”
Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design (2021, methodology): “Following Shin et al. (2021); Akbar et al. (2021), we formulate antibody design as a CDR generation task, conditioned on the framework region.”
Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures (2022, methodology): “In recent years, deep learning methods achieve significant improvement over sampling algorithms for antibody design by using language models to generate protein sequences [5, 43, 41, 3].”