Accelerating Protein Design Using Autoregressive Generative Models

By Adam J. Riesselman, Jung-Eun Shin, Aaron W. Kollasch, Conor McMahon, Elana Simon, Christopher Sander, A. Manglik, A. Kruse, Debora Marks. 2019

45 citations recorded by the source corpus.

A major biomedical challenge is the interpretation of genetic variation and the ability to design functional novel sequences. Since the space of all possible genetic variation is enormous, there is a concerted effort to develop reliable methods that can capture genotype to phenotype maps. State-of-art computational methods rely on models that leverage evolutionary information and capture complex interactions between residues. However, current methods are not suitable for a large number of important applications because they depend on robust protein or RNA alignments. Such applications include genetic variants with insertions and deletions, disordered proteins, and functional antibodies. Ideally, we need models that do not rely on assumptions made by multiple sequence alignments. Here we borrow from recent advances in natural language processing and speech synthesis to develop a generative deep neural network-powered autoregressive model for biological sequences that captures functional constraints without relying on an explicit alignment structure. Application to unseen experimental measurements of 42 deep mutational scans predicts the effect of insertions and deletions while matching state-of-art missense mutation prediction accuracies. We then test the model on single domain antibodies, or nanobodies, a complex target for alignment-based models due to the highly variable complementarity determining regions. We fit the model to a naïve llama immune repertoire and generate a diverse, optimized library of 105 nanobody sequences for experimental validation. Our results demonstrate the power of the ‘alignment-free’ autoregressive model in mutation effect prediction and design of traditionally challenging sequence families.

Subfields: Molecular Biology, Generative Models, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Generative Models

In Computational Biology & Bioinformatics

Citations in the corpus

MSA Transformer (2021, background): “Riesselman et al. (2019) train autoregressive models on MSAs, but discard the alignment, showing that function can be learned from unaligned sequences.”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, methodology): “In [46, 61], autoregressive generative models were trained to predict the functional effect of mutations and generate natural-like proteins.”
ProGen: Language Modeling for Protein Generation (2020, methodology): “Meanwhile sequence-only generative modeling have been attempted recently through residual causal dilated convolutional neural networks (Riesselman et al., 2019) and variational autoencoders (Costello & Martin, 2019).”
Evolutionary context-integrated deep sequence modeling for protein engineering (2020, methodology): “A generative autoregressive model was proposed by Riesselman et al [48] to predict the mutation effects in protein sequence, without the requirement of multiple sequence alignment.”
Language models enable zero-shot prediction of the effects of mutations on protein function (2021, methodology): “[47] proposes using an autoregressive model that does not require the sequences to be aligned.”
Protein sequence design with deep generative models (2021, background): “Riesselman and coauthors applied autoregressive models to generate single domain antibodies (nanobodies) [30].”
Rapid generation of potent antibodies by autonomous hypermutation in yeast (2020, methodology): “0 A computationally designed 200,000-member naïve nanobody CDR3 library was synthesized as an oligonucleotide pool of CDR3 sequences by Agilent, as previously reported (29).”