Sample-efficient Deep Reinforcement Learning for Dialog Control
By Kavosh Asadi, Jason Williams. 2016
21 citations recorded by the source corpus.
Representing a dialog policy as a recurrent neural network (RNN) is attractive because it handles partial observability, infers a latent representation of state, and can be optimized with supervised learning (SL) or reinforcement learning (RL). For RL, a policy gradient approach is natural, but is sample inefficient. In this paper, we present 3 methods for reducing the number of dialogs required to optimize an RNN-based dialog policy with RL. The key idea is to maintain a second RNN which predicts the value of the current policy, and to apply experience replay to both networks. On two tasks, these methods reduce the number of dialogs/episodes required by about a third, vs. standard policy gradient methods.
Subfields: Artificial Intelligence, Social Psychology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shimon Whiteson
- Shangtong Zhang
- Wendelin Böhmer
- Marc-Alexandre Côté
- Xingdi Yuan
- Yuxiong He
- Ruo Yu Tao
- Feng Yan
In Social Psychology
Citations in the corpus
- Generalized Off-Policy Actor-Critic (2019, methodology): “This setting complies with the common training scheme for Mujoco tasks (Lillicrap et al., 2015; Asadi and Williams, 2016).”
- HyperDrive: exploring hyperparameters with POP scheduling (2017, methodology): “We use a model written in Keras [9] and Theano [6] provided by the authors of [4].”
- Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning (2022, background): “Much RL research in dialogue management defines the state and action spaces to be the tokenized language space [1, 13, 19].”
- Towards Solving Text-based Games by Producing Adaptive Action Spaces (2018, background): “Recent attempts at learning to perform all these tasks through one end-to-end model have produced encouraging results (Li et al., 2017; Bordes and Weston, 2016) but so far, the best-performing models still separate these two tasks (Wen et al., 2016; Asadi and Williams, 2016).”