Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation

By Jiaxuan You, Bowen Liu, Rex Ying, Vijay S. Pande, Jure Leskovec. 2018

1,038 citations recorded by the source corpus.

Generating novel graph structures that optimize given objectives while obeying some given underlying rules is fundamental for chemistry, biology and social science research. This is especially important in the task of molecular graph generation, whose goal is to discover novel molecules with desired properties such as drug-likeness and synthetic accessibility, while obeying physical laws such as chemical valency. However, designing models to find molecules that optimize desired properties while incorporating highly complex and non-differentiable rules remains to be a challenging task. Here we propose Graph Convolutional Policy Network (GCPN), a general graph convolutional network based model for goal-directed graph generation through reinforcement learning. The model is trained to optimize domain-specific rewards and adversarial loss through policy gradient, and acts in an environment that incorporates domain-specific rules. Experimental results show that GCPN can achieve 61% improvement on chemical property optimization over state-of-the-art baselines while resembling known molecules, and achieve 184% improvement on the constrained property optimization task.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Molecular Biology, Generative Models, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Molecular Biology

In Generative Models

In Graph Learning

Citations in the corpus

D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, background): “However, downstream applications of generative models are often based on various conditioning signals, such as labels [61], text descriptions [60], reward values [101], or similarity with existing data [45].”
Strategies for Pre-training Graph Neural Networks (2019, background): “ by local protein-protein interaction (PPI) graphs [57, 55]. Supervised learning of graphs, especially with Graph Neural Networks (GNNs) [26, 15, 59, 53], has shown promising results in these domains [64, 56, 13, 60]. Despite the promise, there remain two key challenges in applying GNNs to these sci”
Position-aware Graph Neural Networks (2019, background): “…node classiﬁcation (Kipf & Welling, 2017), link prediction (Grover & Leskovec, 2016), graph classiﬁcation (Ying et al., 2018b) and graph generation (You et al., 2018b), with successful applications across domains such as social and information networks (Ying et al., 2018a), chemistry (You et…”
Learning Multimodal Graph-to-Graph Translation for Molecular Optimization (2018, methodology): “On the QED and DRD2 datasets, our models discover new compounds much more frequently than VSeq2Seq, but less often than MMPA and GCPN. Nonetheless, these methods have much lower translation success rate.”
Self-Supervised Learning: Generative or Contrastive (2020, methodology): “MRNN and GCPN both use a reinforcement learning framework to generate molecule graphs through optimizing domain-speciﬁc rewards.”
Permutation Invariant Graph Generation via Score-Based Generative Modeling (2020, methodology): “Other approaches instead focus on selecting a speciﬁc node ordering based on heuristics: GraphRNN (You et al., 2018b) uses random breadth-ﬁrst search (BFS) to determine an ordering, and GRAN (Liao et al., 2019) adaptively chooses an ordering depending on the input graph from a family of pre-deﬁned…”
Learning Gradient Fields for Molecular Conformation Generation (2021, background): “…of molecules, where nodes represent atoms and edges represent bonds, have been widely applied to a variety of molecular modeling tasks, ranging from property prediction (Gilmer et al., 2017; Hu et al., 2020) to molecule generation (Jin et al., 2018; You et al., 2018; Shi et al., 2020).”
Conditional Structure Generation through Graph Variational Generative Adversarial Nets (2019, background): “However, they either only work with single graphs and fixed sets of nodes [18, 4, 7, 25, 52, 12], or model single representative sets of graphs which essentially belong to the same semantic group [47, 36, 16, 24, 46].”
Graph Representation Learning via Graphical Mutual Information Maximization (2020, methodology): “Recent methods towards this goal, commonly categorized as Graph Neural Networks (GNNs), have made remarkable advancements in a great many learning tasks, such as node classification [35, 42, 50], link prediction [26, 27, 49], graph alignment [12, 19, 43], molecular generation [4, 5, 46], to name a f”
Mol-CycleGAN: a generative model for molecular optimization (2019, methodology): “After 10–20 iterations it reaches very high values of logP observed from molecules which are not drug-like, similarly to those obtained with RL [35].”
MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design (2022, background): “Speciﬁcally, [108], [109], [110], [111] leverage policy gradient and domain-speciﬁc reward function to optimize the graph policy neural network; [103] leverages deep Q-network to conduct value learning, where a deep Q-network is trained to predict the future reward and during each step, the step…”
Geometric deep learning on molecular representations (2021, methodology): “GNNs have also been combined with variational autoencoders [34, 84–86] and reinforcement learning [33, 87, 88].”
Graph Representation Learning in Biomedicine and Healthcare (2021, neutral): “…molecular graphs with desired properties such as drug likeness and synthetic accessibility, while obeying physical laws such as chemical valency 98 ), JT-VAE (a variational autoencoder that generates molecular graphs in two phases, by first generating a tree-structured scaffold over chemical…”
Optimization of Molecules via Deep Reinforcement Learning (2018, background): “The success rate is the percentage of molecules satisfying the similarity constraint. avalues are reported in You et al.18. aThe Tanimoto similarity uses the ratio of the intersecting set to the union set as the measure of similarity.”
GraphQA: protein model quality assessment using graph convolutional networks (2020, background): “…2018a), drug-drug interaction (Zitnik et al., 2018), drug-target interaction (Gao et al., 2018) molecular optimization (Jin et al., 2019), and generation of proteins, molecules and drugs (Ingraham et al., 2019a; You et al., 2018; Liu et al., 2018; Li et al., 2018b; Simonovsky & Komodakis, 2018).”
Benchmarking Graph Neural Networks (2023, methodology): “[92] Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec.”
Toward Robust Long Range Policy Transfer (2021, background): “…learning (RL) has lots of success in various applications, such as game playing (Brockman et al. 2016; Silver et al. 2017; Mnih et al. 2015), robotics control (Tassa et al. 2018; Coumans and Bai 2016–2019), molecule de-sign (You et al. 2018), and computer system optimization (Mao et al. 2019a,b).”
A 3D Generative Model for Structure-Based Drug Design (2022, methodology): “The majority of graph-based models generate molecules in an auto-regressive fashion, i.e., adding atoms or fragments sequentially, which could be implemented based upon VAEs [13], normalizing ﬂows [27], reinforcement learning [35, 14], etc.”