Generating functional protein variants with variational autoencoders

By Alex Hawkins-Hooker, F. Depardieu, Sebastien Baur, Guillaume Couairon, A. Chen, David Bikard. 2020

210 citations recorded by the source corpus.

The design of novel proteins with specified function and controllable biochemical properties is a longstanding goal in bio-engineering with potential applications across medicine and nanotechnology. The vast expansion of protein sequence databases over the last decades provides an opportunity for new approaches which seek to learn the sequence-function relationship directly from natural sequence variation. Advances in deep generative models have led to the successful modelling of diverse kinds of high-dimensional data, from images to molecules, allowing the generation of novel, realistic samples. While deep models trained on protein sequence data have been shown to learn biologically meaningful representations helpful for a variety of downstream tasks, their potential for direct use in protein engineering remains largely unexplored. Here we show that variational autoencoders trained on a dataset of almost 70000 luciferase-like oxidoreductases can be used to generate novel, functional variants of the luxA bacterial luciferase. We propose separate VAE models to work with aligned sequence input (MSA VAE) and raw sequence input (AR-VAE), and offer evidence that while both are able to reproduce patterns of amino acid usage characteristic of the family, the MSA VAE is better able to capture long-distance dependencies reflecting the influence of 3D structure. To validate the practical utility of the models, we used them to generate variants of luxA whose function was tested experimentally. As further evidence of the practicality of these methods for design, we showed that conditional variants of both models could be used to increase the solubility of luxA without disrupting function. Altogether 18/24 of the variants generated using the AR-VAE and 21/23 variants generated using the MSA VAE retained some luminescence activity, despite containing as many as 35 differences relative to any training set sequence. These results demonstrate the feasibility of using deep generative models to explore the space of possible protein sequences and generate useful variants, providing a method complementary to rational design and directed evolution approaches.

Subfields: Genetics, Molecular Biology, Generative Models, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

In Generative Models

In Computational Biology & Bioinformatics

Citations in the corpus

De novo protein design by deep network hallucination (2020, background): “…of sequence 6 , and to identify amino acid sequences that either fit well onto specified backbone structures 7–10 or are conditioned on low-dimensional fold representations 11 ; models tailored to generate sequences and/or structures for specific protein families have also been developed 12–16 .”
Transformer protein language models are unsupervised structure learners (2020, background): “The ability to generate sequences that preserve this information is a necessary condition for generation of biologically active proteins (Hawkins-Hooker et al., 2021).”
Function-guided protein design by deep manifold sampling (2021, methodology): “More recently, VAEs [15, 16] have been used for designing novel enzymes and T-cell receptor proteins, obviating the need for MSA, but they have been largely limited to a single family of proteins.”
Utilizing graph machine learning within drug discovery and development (2021, methodology): “[199] trained multiple variational autoencoders on the amino acid sequences of 70 000 luciferaselike oxidoreductases to generate new functional variants of the D ow naded rom http/academ ic.”
Recent advances in de novo protein design: Principles, methods, and applications (2021, background): “A number of generative models were developed for sequence design, including generative adversarial networks (104), variational autoencoders (105, 106), and graph-based (107, 108) models.”
AlphaDesign: A de novo protein design framework based on AlphaFold (2021, background): “Beyond language models, other types of generative models, such as variational autoencoders [56] and generative adversarial networks [57] have been explored for direct protein sequence generation.”
Protein sequence design with deep generative models (2021, unknown): “Hawkins and coauthors generate functional luciferases from two VAE architectures [26]: 1) by computing the alignment first and training a VAE (MSA VAE) on the aligned sequences and 2) by introducing an autoregressive component to the decoder to learn the unaligned sequences (AR VAE).”
Using AlphaFold for Rapid and Accurate Fixed Backbone Protein Design (2021, methodology): “A variety of neural network architectures have been used including variational autoencoders (Greener et al., 2018; Hawkins-Hooker et al., 2021), deep exploration networks (Linder et al.”