Unified rational protein engineering with sequence-only deep representation learning

By E. C. Alley, Grigory Khimulya, Surojit Biswas, Mohammed Alquraishi, George M. Church. 2019

47 citations recorded by the source corpus.

Rational protein engineering requires a holistic understanding of protein function. Here, we apply deep learning to unlabelled amino acid sequences to distill the fundamental features of a protein into a statistical representation that is semantically rich and structurally, evolutionarily, and biophysically grounded. We show that the simplest models built on top of this unified representation (UniRep) are broadly applicable and generalize to unseen regions of sequence space. Our data-driven approach reaches near state-of-the-art or superior performance predicting stability of natural and de novo designed proteins as well as quantitative function of molecularly diverse mutants. UniRep further enables two orders of magnitude cost savings in a protein engineering task. We conclude UniRep is a versatile protein summary that can be applied across protein engineering informatics.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, methodology): “Sequence-only language models have since been trained through autoregressive or autoencoding self-supervision objectives for both discriminative and generative tasks, for example, using LSTMs or Transformer-based architectures [2, 9, 60, 64].”
Accelerating Protein Design Using Autoregressive Generative Models (2019, background): “Recently, autoregressive models have been applied across protein families to learn general features and predict mutation effects in unaligned protein sequences (Alley et al., 2019; Rives et al., 2019).”
Language models enable zero-shot prediction of the effects of mutations on protein function (2021, methodology): “1 Protein language models In the past few years, a number of groups have developed language models for protein sequences [23, 31, 14, 32, 33, 22, 13, 12].”
Energy-based models for atomic-resolution protein conformations (2020, background): “In parallel, recent work proposes that generative models pre-trained on protein sequences can transfer knowledge to downstream supervised tasks (Bepler & Berger, 2019; Alley et al., 2019; Yang et al., 2019; Rives et al., 2019).”
Machine-learning-guided directed evolution for protein engineering (2018, background): “If accuracy is insufficient, learned encodings may be able to improve performance 74,75 .”
Using deep learning to annotate the protein universe (2019, background): “Using (1) as a general-purpose mapping from sequences to embeddings provides a variety of opportunities beyond the task that the model was initially trained for, including annotation of domains of unknown function and supervised learning on small datasets [19, 26].”
Expanding functional protein sequence spaces using generative adversarial networks (2021, background): “On the other hand, machine learning methods are not limited by the number of sequence variations they can process 14–16 and, instead of depending on a semi-random local search process, infer protein properties 15,17 and function 16,18 directly from the amino-acid sequence.”
Universal Transforming Geometric Network (2019, methodology): “Some possible extensions for UTGN include using pre-trained embedding representations of amino acids like UniRep (Alley et al. (2019)).”
Fast and flexible design of novel proteins using graph neural networks (2019, methodology): “We downloaded from UniParc ( 37 ) a dataset of all protein sequences and corresponding domain definitions, and we extracted from this dataset a list of all unique Gene3D domains.”
Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization (2020, methodology): “…on our smallest 1.7M CPCProt model. as baselines on most tasks; however, we use only 0.9% of the number of embedding parameters of the largest model (Alley et al., 2019) and 8.9% of the number of embedding parameters of the smallest neural network (Bepler & Berger, 2019) (Tables 1-5; Figure 1).”