Molecular de-novo design through deep reinforcement learning

By Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, Hongming Chen. 2017

1,267 citations recorded by the source corpus.

This work introduces a method to tune a sequence-based generative model for molecular de novo design that through augmented episodic likelihood can learn to generate structures with certain specified desirable properties. We demonstrate how this model can execute a range of tasks such as generating analogues to a query structure and generating compounds predicted to be active against a biological target. As a proof of principle, the model is first trained to generate molecules that do not contain sulphur. As a second example, the model is trained to generate analogues to the drug Celecoxib, a technique that could be used for scaffold hopping or library expansion starting from a single molecule. Finally, when tuning the model towards generating compounds predicted to be active against the dopamine receptor type 2, the model generates structures of which more than 95% are predicted to be active, including experimentally confirmed actives that have not been included in either the generative model nor the activity prediction model.Graphical abstract.

Subfields: Artificial Intelligence, Materials Chemistry, Mechanical Engineering, Reinforcement Learning, Computational Chemistry & Drug Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

In Mechanical Engineering

In Reinforcement Learning

In Computational Chemistry & Drug Discovery

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, methodology): “fers ways to facilitate cycle of drug development [297]. It is until recent years that RL methods have been applied in various aspects of de novo design for drug discovery and development. Olivecrona [298] used RL to ﬁne tune the recurrent neural network in order to generate molecules with certain d”
Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation (2018, methodology): “Recently, there has been signiﬁcant advances in applying deep learning models to molecule generation [15, 38, 7, 9, 22, 4, 31, 27, 34, 42].”
Mol2vec: Unsupervised Machine Learning Approach with Chemical Intuition (2018, positive): “10 In parallel, natural language processing (NLP) techniques were adopted to learn from classical features, like molecular FPs, 11 SMILES strings, 12 and graph representations of compounds.”
Opportunities and obstacles for deep learning in biology and medicine (2017, methodology): “These generative models successfully learn the grammar of compound representations, with 94% [326] or nearly 98% [323] of generated SMILES corresponding to valid molecular structures.”
Learning Multimodal Graph-to-Graph Translation for Molecular Optimization (2018, methodology): “We use a trained model from Olivecrona et al. (2017) to assess the probability that a compound is active.”
Graphite: Iterative Generative Modeling of Graphs (2018, background): “Finally, there has been a fair share of recent work for generation of special kinds of graphs, such as parsed trees of source code [29] and SMILES representations for molecules [33].”
GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models (2018, methodology): “Most previous work on discovering molecule structures make use of a expert-crafted sequence representations of molecular graph structures (SMILES) (Olivecrona et al., 2017; Segler et al., 2017; G ´ omez-Bombarelli et al., 2016).”
GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders (2018, positive): “While molecules have an intuitive representation as graphs, the ﬁeld has had to resort to textual representations with ﬁxed syntax, e.g. so-called SMILES strings, to exploit recent progress made in text generation with RNNs (Olivecrona et al., 2017; Segler et al., 2017; G´omez-Bombarelli et al., 201”
Learning Deep Generative Models of Graphs (2018, background): “In addition, we also generated 100,000 samples from each model and evaluate the percentage of well-formatted molecule representations and the percentage of unique novel samples not already seen in the training set following (Segler et al., 2017; Olivecrona et al., 2017).”
Likelihood Ratios for Out-of-Distribution Detection (2019, background): “, 2017) and drug molecules (Olivecrona et al., 2017; Gupta et al., 2018), and text (Jozefowicz et al.”
Deep reinforcement learning for de novo drug design (2017, methodology): “Reinforcement Learning approach for de-novo molecular design was employed in (70) as well.”
Improving Molecular Design by Stochastic Iterative Target Augmentation (2020, methodology): “We note that although Olivecrona et al. (2017) also originally evaluated on the DRD2 property, our setup is more challenging: we allow signiﬁcantly less training data for bootstrapping, and prohibit the use of the ground truth predictor before test time.”
Translation between Molecules and Natural Language (2022, background): “Current work in de novo molecule design primarily focuses on applying existing types of generative models to molecules (Olivecrona et al., 2017; Kell et al., 2020; Krenn et al., 2020; Polykovskiy et al., 2020).”
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models (2018, background): “Representing a molecular structure as a string have been quickly adopted (Jaques et al., 2016; Guimaraes et al., 2017; Kadurin et al., 2017; Olivecrona et al., 2017; Yang et al., 2017; Kang and Cho, 2018; Popova et al., 2018; Putin et al., 2018; Segler et al., 2018) for generative models due to the ”
Hierarchical Generation of Molecular Graphs using Structural Motifs (2020, background): “5 ), where the bioactivity is assessed by a property prediction model from Olivecrona et al. (2017).”
Leveraging molecular structure and bioactivity with chemical language models for de novo drug design (2023, methodology): “There are alternative approaches for CLM development, e.g., model ﬁ ne-tuning (step ii) by reinforcement learning 6,25 .”
Graph convolutional networks for computational drug development and discovery (2019, methodology): “[49] De novo molecular design github.com/MarcusOlivecrona/REINVENT to learn molecular representations for data-driven tasks considering both node and edge features.”
Constrained Graph Variational Autoencoders for Molecule Design (2018, methodology): “More specifically, generating and optimizing chemical molecules has been identified as an important real-world application for this set of techniques [8, 23, 24, 28, 29].”