Low-N protein engineering with data-efficient deep learning

By Surojit Biswas, Grigory Khimulya, E. C. Alley, Kevin M. Esvelt, George M. Church. 2020

433 citations recorded by the source corpus.

Protein engineering has enormous academic and industrial potential. However, it is limited by the lack of experimental assays that are consistent with the design goal and sufficiently high throughput to find rare, enhanced variants. Here we introduce a machine learning-guided paradigm that can use as few as 24 functionally assayed mutant sequences to build an accurate virtual fitness landscape and screen ten million sequences via in silico directed evolution. As demonstrated in two dissimilar proteins, GFP from Aequorea victoria (avGFP) and E. coli strain TEM-1 β-lactamase, top candidates from a single round are diverse and as active as engineered mutants obtained from previous high-throughput efforts. By distilling information from natural protein sequence landscapes, our model learns a latent representation of ‘unnaturalness’, which helps to guide search away from nonfunctional sequence neighborhoods. Subsequent low-N supervision then identifies improvements to the activity of interest. In sum, our approach enables efficient use of resource-intensive high-fidelity assays without sacrificing throughput, and helps to accelerate engineered proteins into the fermenter, field and clinic. A deep-learning-guided approach enables protein engineering using only a small number (‘low N’) of functionally characterized variants of target proteins.

Subfields: Molecular Biology, Efficiency & Compression, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Efficiency & Compression

In Computational Biology & Bioinformatics

Citations in the corpus

De novo protein design by deep network hallucination (2020, methodology): “ The design procedure starts off with picking a random amino acid sequence of a given length L (L = 100 throughout the study), passing it through trRosetta and background networks and calculating the objective F according to Eq.(4).”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, methodology): “Various works have analyzed output embeddings of protein models through dimensionality reduction techniques such as PCA or t-SNE (Elnaggar et al., 2020; Biswas et al., 2020).”
Generating functional protein variants with variational autoencoders (2020, background): “In principle such models would make it possible to leverage the full variety of protein sequence information by, for example, pretraining models on entire protein sequence databases [30, 37, 38, 61]; fine-tuning of such models on individual families has shown promise for some prediction tasks and ma”
Protein sequence design with a learned potential (2022, neutral): “Recent experimentally validated efforts for machine learning-based sequence generation have focused on sequence representation learning without structural information, requiring ﬁ tting to data from experiments or from known protein families to produce functional designs 23,24 .”
Language models enable zero-shot prediction of the effects of mutations on protein function (2021, methodology): “Fine-tuning on MSAs has been used in previous work [23, 16] as a stage in transfer learning to specialize a pre-trained model to a protein family, before applying supervision with labeled data.”
AdaLead: A simple and robust adaptive greedy search algorithm for sequence design (2020, background): “It’s important to note that the focus of our study is distinct from those focused on better local approximate models (e.g. (Rao et al., 2019; Riesselman et al., 2017)), or semi-supervised and transfer learning approaches (e.g. (Madani et al., 2020; Biswas et al., 2020)).”
Efficient evolution of human antibodies from general protein language models (2023, positive): “Our techniques can also be used in conjunction with supervised approaches 9,28,33,34,51–54 , and supervising a model over multiple experimental rounds might ultimately lead to higher fitness.”
AlphaDesign: A de novo protein design framework based on AlphaFold (2021, methodology): “A number of works explore re-using previously trained predictors of protein structure or function as parts of an in silico screening framework [69–76].”
Protein Representation Learning by Geometric Structure Pretraining (2022, background): “These works share the common goal of learning informative protein representations that can beneﬁt various downstream applications, like predicting protein function (Rives et al., 2021) and protein-protein interaction (Wang et al., 2019), as well as designing protein sequences (Biswas et al., 2021).”
Guided Generative Protein Design using Regularized Transformers (2022, methodology): “Using this trained model as a way to evaluate subsequent candidate sequences in silico , it is then possible to optimize sequences through search and screen reﬁnement process as in Yang et al. [2019], Biswas et al. [2021].”
Exploring evolution-based & -free protein language models as protein function predictors (2022, background): “Our work points out both strengths and weakness of three SOTA PLMs, which are potentially useful to many downstream biological tasks, including but not limited to protein engineering [6], cancer early detection [14], and drug discovery [21].”
Expanding functional protein sequence spaces using generative adversarial networks (2021, background): “Successful applications of deep learning to in silico directed evolution exemplify the complementarity of traditional protein engineering and machine learning-driven synthetic biology 22,23 .”
Deep learning methods for designing proteins scaffolding functional sites (2021, methodology): “As a test, we used the catalytic sidechain geometry of 5 -3-ketosteroid isomerase (1QJG: residues 14, 38, 99), which catalyzes the isomerization of 5 - to 4 -3-ketosteroid needed for synthesis of steroid hormones in mammals ( 28 ).”
FLIP: Benchmark tasks in fitness landscape inference for proteins (2021, background): “Recent efforts in sequence-function prediction [15, 16] have sought to leverage the information in these unlabeled sequences through pretraining and fine-tuning, and have successfully engineered proteins with brighter fluorescence and high catalytic efficiency [17].”