Unified rational protein engineering with sequence-based deep representation learning

By E. C. Alley, Grigory Khimulya, Surojit Biswas, Mohammed Alquraishi, George M. Church. 2019

1,084 citations recorded by the source corpus.

Rational protein engineering requires a holistic understanding of protein function. Here, we apply deep learning to unlabeled amino-acid sequences to distill the fundamental features of a protein into a statistical representation that is semantically rich and structurally, evolutionarily and biophysically grounded. We show that the simplest models built on top of this unified representation (UniRep) are broadly applicable and generalize to unseen regions of sequence space. Our data-driven approach predicts the stability of natural and de novo designed proteins, and the quantitative function of molecularly diverse mutants, competitively with the state-of-the-art methods. UniRep further enables two orders of magnitude efficiency improvement in a protein engineering task. UniRep is a versatile summary of fundamental protein features that can be applied across protein engineering informatics. UniRep learns fundamental protein features from millions of amino-acid sequences using a recurrent neural network. This summary of features can then be used for protein engineering.

Subfields: Molecular Biology, Representation & Self-Supervised Learning, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Representation & Self-Supervised Learning

In Computational Biology & Bioinformatics

Citations in the corpus

Evaluating Large Language Models Trained on Code (2021, background): “, 2020), biology (Alley et al., 2019; Rives et al., 2021), and even across multiple modalities (Das et al.”
High-resolution de novo structure prediction from primary sequence (2022, methodology): “Similar to the ESM-1b model ( 12 ), the overall architecture of OmegaPLM is a self-attention model ( 24 ) in which each token is a single amino acid.”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, methodology): “Sequence-only language models have since been trained through autoregressive or autoencoding self-supervision objectives for discriminative and generative tasks, for example, using LSTMs or Transformer-based architectures (Alley et al., 2019; Bepler & Berger, 2019; Rao et al., 2019; Rives et al., 20”
Evolutionary context-integrated deep sequence modeling for protein engineering (2020, methodology): “Following a previous work [27] , we also used the four best performing models as chosen in Yang et al. [46] , including the original model (k=3, w=7), the scrambled model (k=3, w=5), the random model (k=3, w=7), and the uniform model (k=4, w=1).”
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing (2020, background): “Recently, the leap of NLP through advanced LMs have successfully been generalized toward understanding the language of life through advanced LMs trained on proteins [16, 17, 18, 19, 20, 21, 22, 23, 24].”
Transformer protein language models are unsupervised structure learners (2020, methodology): “Lastly, Alley et al. (2019) presented a method of fine-tuning where a pretrained language model is further trained on the MSA of the sequence of interest (‘evotuning’).”
A guide to machine learning for biologists (2021, background): “biological sequences Protein engineering [68] Variable input size Long training times”