Learning Robust Dialog Policies in Noisy Environments

By Maryam Fazel-Zarandi, Shang-Wen Li, Jin Cao, Jared Casale, Peter Henderson, David Whitney, Alborz Geramifard. 2017

24 citations recorded by the source corpus.

Modern virtual personal assistants provide a convenient interface for completing daily tasks via voice commands. An important consideration for these assistants is the ability to recover from automatic speech recognition (ASR) and natural language understanding (NLU) errors. In this paper, we focus on learning robust dialog policies to recover from these errors. To this end, we develop a user simulator which interacts with the assistant through voice commands in realistic scenarios with noisy audio, and use it to learn dialog policies through deep reinforcement learning. We show that dialogs generated by our simulator are indistinguishable from human generated dialogs, as determined by human evaluators. Furthermore, preliminary experimental results show that the learned policies in noisy environments achieve the same execution success rate with fewer dialog turns compared to fixed rule-based policies.

Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jason Aaron Edward Weston
2. Kurt Shuster
3. Y-Lan Boureau
4. Eric Michael Smith
5. Emily Dinan
6. Arthur Szlam
7. Mohit Bansal
8. Jianfeng Gao

In Natural Language Processing

1. Jason Aaron Edward Weston
2. Kurt Shuster
3. Y-Lan Boureau
4. Emily Dinan
5. Eric Michael Smith
6. Arthur Szlam
7. Mohit Bansal
8. Stephen Roller

In Reinforcement Learning

1. Joelle Pineau
2. Arthur Szlam
3. Kurt Shuster
4. Emily Dinan
5. Jason Aaron Edward Weston
6. Peter Henderson
7. Jack Urbanek
8. Antoine Bordes

Citations in the corpus

• Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “This mirrors other successes in using selfplay, self-chat, and simulated users to evaluate dialogue systems (Fazel-Zarandi et al., 2017; Shah et al., 2018a,b; Wei et al., 2018; Ghandeharioun et al., 2019).”
• Recipes for Building an Open-Domain Chatbot (2020, methodology): “This mirrors other successes in using self-play, self-chat, and simulated users to evaluate dialogue systems (Fazel-Zarandi et al., 2017; Shah et al., 2018a,b; Wei et al., 2018; Ghandeharioun et al., 2019).”
• A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, background): “Recent advances in dialogue management have involved using reinforcement learning methods to learn a dialogue management policy (Singh et al., 2002; Scheffler and Young, 2002; Pietquin et al., 2011; Rieser and Lemon, 2011; Peng et al., 2017; Fazel-Zarandi et al., 2017).”
• An Introduction to Deep Reinforcement Learning (2018, unknown): “Deep RL has also shown lots of potential for real-world applications such as robotics (Kalashnikov et al., 2018), self-driving cars (You et al., 2017), finance (Deng et al., 2017), smart grids (François-Lavet et al., 2016b), dialogue systems (Fazel-Zarandi et al., 2017), etc.”