Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models
By Gabriel Lima Guimaraes, Benjamín Sánchez-Lengeling, Pedro Luis Cunha Farias, Alán Aspuru-Guzik. 2017
580 citations recorded by the source corpus.
In unsupervised data generation tasks, besides the generation of a sample based on previous observations, one would often like to give hints to the model in order to bias the generation towards desirable metrics. We propose a method that combines Generative Adversarial Networks (GANs) and reinforcement learning (RL) in order to accomplish exactly that. While RL biases the data generation process towards arbitrary metrics, the GAN component of the reward function ensures that the model still remembers information learned from data. We build upon previous results that incorporated GANs and RL in order to generate sequence data and test this model in several settings for the generation of molecules encoded as text sequences (SMILES) and in the context of music generation, showing for each case that we can effectively bias the generation process towards desired metrics.
Subfields: Artificial Intelligence, Materials Chemistry, Reinforcement Learning, Generative Models, Natural Language Generation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Percy Liang
- Jianfeng Gao
- Jure Leskovec
- Quoc V. Le
- Caiming Xiong
- Dacheng Tao
In Materials Chemistry
- Tommi S. Jaakkola
- Regina Barzilay
- Jure Leskovec
- Wengong Jin
- José Miguel Hernández-Lobato
- Ola Engkvist
- Rex Ying
- Jiaxuan You
In Reinforcement Learning
- José Miguel Hernández-Lobato
- Jure Leskovec
- Rex Ying
- Jiaxuan You
- Tommi S. Jaakkola
- Regina Barzilay
- Wengong Jin
- Caiming Xiong
In Generative Models
- Tommi S. Jaakkola
- Regina Barzilay
- Wengong Jin
- Jure Leskovec
- William L. Hamilton
- Rex Ying
- Alán Aspuru-Guzik
- Jiaxuan You
In Natural Language Generation
- José Miguel Hernández-Lobato
- Yongjun Hong
- Uiwon Hwang
- Sungroh Yoon
- Thomas Kipf
- Nicola De Cao
- Jaeyoon Yoo
- Alán Aspuru-Guzik
Citations in the corpus
- Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation (2018, background): “Note that both scores are calculated from empirical prediction models whose parameters are estimated from related datasets [41, 1], and these scores are widely used in previous molecule generation papers [9, 22, 4, 39, 27].”
- Learning Multimodal Graph-to-Graph Translation for Molecular Optimization (2018, background): “…through generative modeling (G´omez-Bombarelli et al., 2016; Segler et al., 2017; Kusner et al., 2017; Dai et al., 2018; Jin et al., 2018; Samanta et al., 2018; Li et al., 2018a) and reinforcement learning (Guimaraes et al., 2017; Olivecrona et al., 2017; Popova et al., 2018; You et al., 2018a).”
- Molecular de-novo design through deep reinforcement learning (2017, methodology): “Guimaraes et al. demonstrates how such a method can be combined with a fixed scoring function for molecular de novo design [32].”
- Improving Abstraction in Text Summarization (2018, background): “A recent approach to training sequential models utilizes generative adversarial networks to improving the human perceived quality of generated outputs (Fedus et al., 2018; Guimaraes et al., 2017; Liu et al., 2018).”
- Syntax-Directed Variational Autoencoder for Structured Data (2018, background): “Techniques including data augmentation (Bjerrum, 2017), active learning (Janz et al., 2017) and reinforcement learning (Guimaraes et al., 2017) also been proposed to tackle this issue.”
- Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations (2021, methodology): “It does not require any complex optimization over latent space compared with existing methods for controlled generation, such as BO 30,55 , RL 38,39 or SS generative models 40 .”
- Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery (2018, methodology): “Benhenda [1] aimed at producing SMILES strings with ORGAN [8] and RL that are active against the dopamine receptor D2 (DRD2).”
- Generating 3D Molecular Structures Conditional on a Receptor Binding Site with Deep Generative Models (2020, methodology): “GANs have also been applied to molecular generative models and used with reinforcement learning to successfully generate novel molecules with desired properties [34, 35].”
- MolGAN: An implicit generative model for small molecular graphs (2018, methodology): “We then compare MolGAN with ORGAN (Guimaraes et al., 2017) since it is the most related work to ours: ORGAN is a sequential generative model operating on SMILES representations, optimizing towards several chemical properties with an RL objective (see Section 5.2).”