Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learning

By Alexandros Papangelis, Yi-Chia Wang, Piero Molino, Gökhan Tür. 2019

40 citations recorded by the source corpus.

Some of the major challenges in training conversational agents include the lack of large-scale data of real-world complexity, defining appropriate evaluation measures, and managing meaningful conversations across many topics over long periods of time. Moreover, most works tend to assume that the conversational agent’s environment is stationary, a somewhat strong assumption. To remove this assumption and overcome the lack of data, we take a step away from the traditional training pipeline and model the conversation as a stochastic collaborative game. Each agent (player) has a role (“assistant”, “tourist”, “eater”, etc.) and their own objectives, and can only interact via language they generate. Each agent, therefore, needs to learn to operate optimally in an environment with multiple sources of uncertainty (its own LU and LG, the other agent’s LU, Policy, and LG). In this work, we present the first complete attempt at concurrently training conversational agents that communicate only via self-generated language and show that they outperform supervised and deep learning baselines.

Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “Other efforts in the software space include RASA (Bocklisch et al., 2017), PolyAI (Henderson et al., 2019), Uber’s Plato (Papangelis et al., 2019), Microsoft’s IceCaps (Shiv et al., 2019), and Huggingface Transformers library (Wolf et al., 2019a), not focused on dialogue per se, but used as a base i”
Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems (2019, background): “Our goal is to make this work transparent and reproducible, while adding diversity to the platforms future practitioners can choose to use (e.g. ParlAI [34], Plato Research Dialog System [35], ChatEval [36]).”
Survey on reinforcement learning for language processing (2021, background): “In [90, 89], a complete attempt at concurrently training conversational agents is presented.”