Tuning Recurrent Neural Networks with Reinforcement Learning

By Natasha Jaques, Shixiang Shane Gu, Richard E. Turner, Douglas Eck. 2016

113 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Learning to summarize from human feedback (2020, background): “There is also a rich literature on using RL to optimize automatic metrics for NLP tasks, such as ROUGE for summarization [50, 65, 45, 15, 19], BLEU for translation [50, 66, 1, 43], and other domains [61, 27, 26].”
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting (2022, methodology): “These preferences could include topic and sentiment (Dathathri et al., 2020), valid musical notes and molecular structures (Jaques et al., 2017a), code compilability (Korbak et al., 2021), reducing distributional biases (Khalifa et al., 2021; Weidinger et al., 2021), evaluation metrics for Machine…”
Grammar Variational Autoencoder (2017, background): “Generative machine learning models have been used recently to produce extraordinary results, from realistic musical improvisation (Jaques et al., 2016), to changing facial expressions in images (Radford et al., 2015; Upchurch et al., 2016), to creating realistic looking artwork (Gatys et al., 2015).”
Molecular de-novo design through deep reinforcement learning (2017, methodology): “Through learning an augmented episodic likelihood which is a composite of prior likelihood [17] and a user defined scoring function, the method aims to fine-tune an RNN pre-trained on the ChEMBL database [20] towards generating desirable compounds.”
Synthetic Data for Deep Learning (2019, methodology): “To then use the models to obtain molecules with desired properties, researchers used either a small set of labeled positive examples [530] or augment the RNN training procedure with reinforcement learning [280].”
Deep Reinforcement Learning: An Overview (2017, background): “Being more practical, we expect more applications/extensions of techniques in Silver et al. (2016) in solving problems requiring titanic search spaces, like classical AI problems, e.g., planning, scheduling, and constraint satisfaction, etc.”
Deep Learning Techniques for Music Generation - A Survey (2017, methodology): “52 shows the evolution during the training phase of the two types of rewards (adherence to Note RNN and to music theory), with three different reinforcement learning algorithms: Q-learning, Ψ -learning and G-learning (see details in [86]).”
MidiNet: A Convolutional Generative Adversarial Network for Symbolic-Domain Music Generation (2017, methodology): “This includes models for generating a melody sequence or audio waveforms by following a few priming notes [10,18,19,22,31,33], accompanying a melody sequence with music of other parts [15], or playing a duet with human [4,26].”
MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment (2017, background): “are usually limited in certain aspects. Examples include: enforcing temporal structure with the more expensive recurrent neural network (RNN) design (Mogren 2016), generating only single-track music (Jaques et al. 2016; Roberts et al. 2016; Bretan, Weinberg, and Heck 2016; Yang, Chou, and Yang 2017)”
MuseGAN: Symbolic-domain Music Generation and Accompaniment with Multi-track Sequential Generative Adversarial Networks (2017, background): “Examples include: enforcing temporal structure with the more expensive recurrent neural network (RNN) design (Mogren 2016), generating only single-track music (Jaques et al. 2016; Roberts et al. 2016; Bretan, Weinberg, and Heck 2016; Yang, Chou, and Yang 2017), requiring user input (Chu, Urtasun, an”
Energy-Based Models for Code Generation under Compilability Constraints (2021, methodology): “This approach, termed “conservative ﬁne-tuning”, was applied to generating melodies with music theory rewards and organic molecules with synthesizabil-ity rewards by Jaques et al. (2017) as well ﬁne-tuning language models for controllable language generation by Ziegler et al. (2019).”
Distributional Reinforcement Learning for Energy-Based Sequential Models (2019, background): “[7] and [13] attempt to combine log-likelihood (aka perplexity) and reward-based training in a more integrated way.”