De novo protein design by deep network hallucination

By Ivan V. Anishchenko, S. Pellock, T. M. Chidyausiku, T. Ramelot, Sergey Ovchinnikov, J. Hao, Khushboo Bafna, Christoffer H Norn, A. Kang, A. Bera and 5 more. 2020

559 citations recorded by the source corpus.

There has been considerable recent progress in protein structure prediction using deep neural networks to predict inter-residue distances from amino acid sequences1–3. Here we investigate whether the information captured by such networks is sufficiently rich to generate new folded proteins with sequences unrelated to those of the naturally occurring proteins used in training the models. We generate random amino acid sequences, and input them into the trRosetta structure prediction network to predict starting residue–residue distance maps, which, as expected, are quite featureless. We then carry out Monte Carlo sampling in amino acid sequence space, optimizing the contrast (Kullback–Leibler divergence) between the inter-residue distance distributions predicted by the network and background distributions averaged over all proteins. Optimization from different random starting points resulted in novel proteins spanning a wide range of sequences and predicted structures. We obtained synthetic genes encoding 129 of the network-‘hallucinated’ sequences, and expressed and purified the proteins in Escherichia coli; 27 of the proteins yielded monodisperse species with circular dichroism spectra consistent with the hallucinated structures. We determined the three-dimensional structures of three of the hallucinated proteins, two by X-ray crystallography and one by NMR, and these closely matched the hallucinated models. Thus, deep networks trained to predict native protein structures from their sequences can be inverted to design new proteins, and such networks and methods should contribute alongside traditional physics-based models to the de novo design of proteins with new functions. The trRosetta neural network was used to iteratively optimise model proteins from random 100-amino-acid sequences, resulting in ‘hallucinated’ proteins, which when expressed in bacteria closely resembled the model structures.

Subfields: Materials Chemistry, Molecular Biology, Generative Models, Computational Chemistry & Drug Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Materials Chemistry

In Molecular Biology

In Generative Models

In Computational Chemistry & Drug Discovery

Citations in the corpus

Accurate prediction of protein structures and interactions using a 3-track neural network (2021, positive): “Intrigued by the DeepMind results, and with the goal of increasing protein structure prediction accuracy for structural biology research and advancing protein design (4), we explored network architectures incorporating different combinations of these five properties.”
AlphaFold2 and its applications in the fields of biology and medicine (2023, methodology): “111,112 The integrity of the structures predicted is validated and con ﬁ rmed by standard ab initio folding, protein structure analysis methods, and rigorous all-atom molecular dynamics simulations.”
Learning inverse folding from millions of predicted structures (2022, background): “Anishchenko et al. (2021) Structure-agnostic protein sequence design We point the reader to Wu et al. (2021) for a review of the many machine learning-based sequence design approaches that do not explicitly model protein structures.”
Protein sequence design with a learned potential (2022, methodology): “…now possible to build models that learn complex functions of protein sequence and structure, including models for protein backbone generation 18 – 20 and protein structure prediction 21,22 ; as a result, we were curious as to whether an entirely learned method could be used to design protein…”
Protein sequence design by conformational landscape optimization (2021, methodology): “We benchmarked this optimization method by redesigning 2,000 proteins previously generated using Markov chain Monte Carlo optimization with trRosetta (15).”
Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem (2022, neutral): “Other authors use neural net [36, 2, 38, 19, 39], but require a computationally challenging conformational landscape exploration.”
Scaffolding protein functional sites using deep learning (2022, methodology): “We refer to this process as “hallucination” as it produces solutions that the network considers ideal proteins but do not correspond to any known natural protein; crystal and NMR structures confirm that the hallucinated sequences fold to the hallucinated structures (12).”
Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models (2022, unknown): “…information or through external “potentials”, and (3) unlike methods that design proteins through generation or optimization of protein sequences alone 1,3,4,10,11 , DDPMs can be formulated to generate protein structures directly, enabling more direct control over structural properties.”
Protein tertiary structure prediction and refinement using deep learning and Rosetta in CASP14 (2021, unknown): “Following Anishchenko et al, 14 we estimated the quality of these predictions on a per residue pair basis by calculating the KL-divergence ( D KL, ijk , indices i , j define a residue pair and index k enumerates MSAs which were used to get the predictions) of the predicted distance and orientation d”
Improving de novo protein binder design with deep learning (2022, unknown): “…multiple sequence alignments (which contain rich co-evolutionary information on residues likely to be in contact, etc), but for de novo designedsequences,whicharegenerallymorestableandmoreregular than naturally occurring proteins, accurate predictions can be obtained from single sequences 18, 19 .”
A guide to machine learning for biologists (2021, background): “This will also assist in the move from predictive machine learning to generative models that can create new entities, such as designing proteins with novel structures and functions 104,105 .”
Recent advances in de novo protein design: Principles, methods, and applications (2021, methodology): “Moreover, the most recent developments of deep learning for protein structure prediction (41 – 43) foreshadow new methods in the design, taking advantage of learned principles of the protein structure (44, 45).”
Hallucinating protein assemblies (2022, methodology): “…exploration of the space of possible structures has become possible by deep network hallucination: starting from a random amino acid sequence, Monte Carlo optimization with a loss function favoring folding to a well-defined state converges on new sequences that fold to new structures ( 18 – 21 ).”
De novo protein design by inversion of the AlphaFold structure prediction network (2022, methodology): “This method allows the network to “ hallucinate ” a sequence and structure different from those in nature (Anishchenko et al., 2021).”