Deep neural language modeling enables functional protein generation across families

By Ali Madani, Ben Krause, E. Greene, Subu Subramanian, Benjamin P. Mohr, J. Holton, J. L. Olmos, Caiming Xiong, Zachary Z. Sun, Richard Socher and 2 more. 2021

55 citations recorded by the source corpus.

Bypassing nature’s evolutionary trajectory, de novo protein generation—defined as creating artificial protein sequences from scratch—could enable breakthrough solutions for biomedical and environmental challenges. Viewing amino acid sequences as a language, we demonstrate that a deep learning-based language model can generate functional artificial protein sequences across families, akin to generating grammatically and semantically correct natural language sentences on diverse topics. Our protein language model is trained by simply learning to predict the next amino acid for over 280 million protein sequences from thousands of protein families, without biophysical or coevolutionary modeling. We experimentally evaluate model-generated artificial proteins on five distinct antibacterial lysozyme families. Artificial proteins show similar activities and catalytic efficiencies as representative natural lysozymes, including hen egg white lysozyme, while reaching as low as 44% identity to any known naturally-evolved protein. The X-ray crystal structure of an enzymatically active artificial protein recapitulates the conserved fold and positioning of active site residues found in natural proteins. We demonstrate our language model’s ability to be adapted to different protein families by accurately predicting the functionality of artificial chorismate mutase and malate dehydrogenase proteins. These results indicate that neural language models successfully perform de novo protein generation across protein families and may prove to be a tool to shortcut evolution.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

Learning inverse folding from millions of predicted structures (2022, background): “, 2021), or model sequences directly (Rives et al., 2021; Madani et al., 2021; Shin et al., 2021; Gligorijevic et al., 2021; Bryant et al., 2021; Dallago et al., 2021).”
Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies (2022, methodology): “Auto-regressive transformer models have been used to 30 generate functional proteins entirely from sequence learning (24).”
Evolutionary velocity with protein language models (2021, unknown): “Promising advances in machine learning have improved the ability of a class of algorithms called language models to learn the rules that govern how amino acids can appear together to form a protein sequence (Alley et al., 2019; Bepler and Berger, 2019, 2021; Hie et al., 2021; Hsu et al., 2022; Rao e”
ProGen2: Exploring the Boundaries of Protein Language Models (2022, methodology): “In prior work, ﬁnetuning the ProGen model on a set of lysozyme families yielded proteins retaining functional behavior, and even rivaling that of a natural hen egg white lysozyme (Madani et al., 2021).”
AlphaDesign: A de novo protein design framework based on AlphaFold (2021, background): “[59] have expanded on [55] by additionally ﬁne-tuning the generative model on a speciﬁc protein family or function.”