Transfer Learning for User Adaptation in Spoken Dialogue Systems

By Aude Genevay, Romain Laroche. 2016

39 citations recorded by the source corpus.

This paper focuses on user adaptation in Spoken Dialogue Systems. It is considered that the system has already been optimised with Reinforcement Learning methods for a set of users. The goal is to use and transfer this prior knowledge to adapt the system to a new user as quickly as possible without impacting asymptotic performance. The first contribution is a source selection method using a multi-armed stochastic bandit algorithm in order to improve the jumpstart, \emph{i.e.} the average performance at the start of the learning curve. Contrarily to previous source selection methods, there is no need to define a metric between users, and it is parameter free. The second contribution is an innovative method for selecting the most informative transitions within the previously selected source, to improve the target model, in such a way that only transitions that were not observed with the target user are transferred from the selected source. For our experimentation, Reinforcement Learning is performed with the Fitted $Q$-Iteration algorithm. Both methods are validated on a negotiation game: an appointment scheduling simulator that allows the definition of simulated user models adopting diversified behaviours. Compared to state-of-the-art transfer algorithms, results show significant improvements for both jumpstart and asymptotic performance.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Information Systems

Citations in the corpus

A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, methodology): “…or reinforcement learning framework) using transfer learning (Fabbrizio et al., 2004; Forgues et al., 2014; Serban and Pineau, 2015; Serban et al., 2016; Lowe et al., 2015a; Vandyke et al., 2015; Wen et al., 2016; Gašić et al., 2016; Mo et al., 2016; Genevay and Laroche, 2016; Chen et al., 2016)”
Domain Adaptive Dialog Generation via Meta Learning (2019, background): “Mo et al. (2018) and Genevay and Laroche (2016) also trains a user adaptive dialog systems using transfer learning.”
Personalizing a Dialogue System With Transfer Reinforcement Learning (2016, methodology): “Genevay and Laroche [8] propose to select and transfer an optimized policy from source users to a target user by using a multi-armed stochastic bandit algorithm which does not require a predeﬁned user similarity measure.”
Neural personalized response generation as domain adaptation (2017, background): “Genevay and Laroche [12] presented a source selection approach and a transition selection approach to overcome the cold start problem for the new coming users of spoken dialogue systems.”
Transfer Reinforcement Learning with Shared Dynamics (2017, background): “…money), and Transfer Learning for RL (Taylor and Stone 2009; Lazaric 2012), through reuse of knowledge acquired from similar tasks, has proven useful in many RL domains: Atari games (Romoff, Bengio, and Pineau 2016), robotics (Taylor, Stone, and Liu 2007), or dialogue (Genevay and Laroche 2016).”
Fine Grained Knowledge Transfer for Personalized Task-oriented Dialogue Systems (2017, background): “Learning based task-oriented dialogue systems (Casanueva et al. 2015; Genevay and Laroche 2016; Gaˇsic et al. 2013; Gasic et al. 2014; Williams and Zweig 2016; Bordes and Weston 2016) can select appropriate answers from a set of predeﬁned answers.”