Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning
By Julia Kiseleva, Maarten de Rijke. 2018
55 citations recorded by the source corpus.
The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Shixiang Shane Gu
- Natasha Jaques
- Judy Hanwen Shen
- Asma Ghandeharioun
- Àgata Lapedriza
- Rosalind W. Picard
- Craig Ferguson
In Computer Vision and Pattern Recognition
- Shixiang Shane Gu
- Sergey Levine
- Judy Hanwen Shen
- Asma Ghandeharioun
- Natasha Jaques
- Àgata Lapedriza
- Rosalind W. Picard
- Craig Ferguson
In Natural Language Processing
- Shixiang Shane Gu
- Judy Hanwen Shen
- Asma Ghandeharioun
- Natasha Jaques
- Àgata Lapedriza
- Craig Ferguson
- Noah J. Jones
- Rosalind W. Picard
Citations in the corpus
- Human-centric Dialog Training via Offline Reinforcement Learning (2020, background): “RL in the open-domain dialog setting is less explored (Li et al., 2016, 2017b, 2018).”
- Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation (2020, background): “To address this problem, reward function learning that relies on expert demonstrations has been introduced (Takanobu et al., 2019; Li et al., 2019b).”
- Hierarchical Reinforcement Learning for Open-Domain Dialog (2019, background): “GAN-like discriminator trained to distinguish human and model generated text [24, 25, 55].”