Adversarial Learning of Task-Oriented Neural Dialog Models
40 citations recorded by the source corpus.
In this work, we propose an adversarial learning method for reward estimation in reinforcement learning (RL) based task-oriented dialog models. Most of the current RL based task-oriented dialog systems require the access to a reward signal from either user feedback or user ratings. Such user ratings, however, may not always be consistent or available in practice. Furthermore, online dialog policy learning with RL typically requires a large number of queries to users, suffering from sample efficiency problem. To address these challenges, we propose an adversarial learning method to learn dialog rewards directly from dialog samples. Such rewards are further used to optimize the dialog policy with policy gradient based RL. In the evaluation in a restaurant search domain, we show that the proposed adversarial dialog learning method achieves advanced dialog success rate comparing to strong baseline methods. We further discuss the covariate shift problem in online adversarial dialog learning and show how we can address that with partial access to user feedback.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Minlie Huang
- Michel Galley
- Lihong Li
- Jason Aaron Edward Weston
- Mohit Bansal
- Yejin Choi
- Percy Liang
Citations in the corpus
- Neural Approaches to Conversational AI (2018, positive): “b) used multi-objective RL to automatically learn how to linearly combine multiple metrics of interest in the definition of reward function. Finally, inspired by adversarial training in deep learning, Liu and Lane (2018) proposed to view the reward function as a discriminator that distinguishes dialo”
- Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation (2020, methodology): “We compare Act-VRNN with three policy learning baselines: (1) PPO (Schulman et al., 2017) us-ing hand-crafted rewards setting; (2) ALDM (Liu and Lane, 2018); (3) GDPL (Takanobu et al., 2019); We further consider using semi-supervised techniques to enhance the baselines under semi-supervised…”
- Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog (2019, background): “Taxi domain has the least slots for constraints and requests, which makes it easier to learn a reward about that domain, thus leading ALDM to a local optimal.”