Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue

By Dongyeop Kang, Anusha Balakrishnan, Pararth Shah, Paul A. Crook, Y-Lan Boureau, Jason Aaron Edward Weston. 2019

125 citations recorded by the source corpus.

Traditional recommendation systems produce static rather than interactive recommendations invariant to a user’s specific requests, clarifications, or current mood, and can suffer from the cold-start problem if their tastes are unknown. These issues can be alleviated by treating recommendation as an interactive dialogue task instead, where an expert recommender can sequentially ask about someone’s preferences, react to their requests, and recommend more appropriate items. In this work, we collect a goal-driven recommendation dialogue dataset (GoRecDial), which consists of 9,125 dialogue games and 81,260 conversation turns between pairs of human workers recommending movies to each other. The task is specifically designed as a cooperative game between two players working towards a quantifiable common goal. We leverage the dataset to develop an end-to-end dialogue system that can simultaneously converse and recommend. Models are first trained to imitate the behavior of human players without considering the task goal itself (supervised training). We then finetune our models on simulated bot-bot conversations between two paired pre-trained models (bot-play), in order to achieve the dialogue goal. Our experiments show that models finetuned with bot-play learn improved dialogue strategies, reach the dialogue goal more often when paired with a human, and are rated as more consistent by humans compared to models trained without bot-play. The dataset and code are publicly available through the ParlAI framework.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “This is a continuation of the approach taken in multiple papers using role-played situations (Rashkin et al., 2019), assigned traits (Shuster et al., 2018, 2019), assigned movie preferences (Kang et al., 2019), or even an entire fantasy universe (Urbanek et al., 2019; Prabhumoye et al., 2020).”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “Therefore, it is natural to incorporate RL into the CRS framework [160, 88, 89, 166, 140, 76].”
CRSLab: An Open-Source Toolkit for Building Conversational Recommender System (2021, methodology): “, 2019) 13,802 91,209 Movie, Book Path Generation DB CNet GoRecDial (Kang et al., 2019) 9,125 170,904 Movie Action Prediction DB CNet DuRecDial (Liu et al.”
A Survey on Conversational Recommender Systems (2020, methodology): “GoRecDial This dataset consists of 9,125 dialogue interactions and 81,260 conversation turns collected through pairs of human workers; here also one plays the role of a movie seeker and the other as a recommender [63].”