Personalizing a Dialogue System With Transfer Reinforcement Learning

By Kaixiang Mo, Yu Zhang, Shuangyin Li, Jiajun Li, Qiang Yang. 2016

97 citations recorded by the source corpus.

It is difficult to train a personalized task-oriented dialogue system because the data collected from each individual is often insufficient. Personalized dialogue systems trained on a small dataset is likely to overfit and make it difficult to adapt to different user needs. One way to solve this problem is to consider a collection of multiple users as a source domain and an individual user as a target domain, and to perform transfer learning from the source domain to the target domain. By following this idea, we propose a PErsonalized Task-oriented diALogue (PETAL) system, a transfer reinforcement learning framework based on POMDP, to construct a personalized dialogue system. The PETAL system first learns common dialogue knowledge from the source domain and then adapts this knowledge to the target domain. The proposed PETAL system can avoid the negative transfer problem by considering differences between the source and target users in a personalized Q-function. Experimental results on a real-world coffee-shopping data and simulation data show that the proposed PETAL system can learn optimal policies for different users, and thus effectively improve the dialogue quality under the personalized setting.

Subfields: Artificial Intelligence, Information Systems, Reinforcement Learning, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Reinforcement Learning

In Dialogue & Conversational AI

Citations in the corpus

Story Ending Prediction by Transferable BERT (2019, neutral): “Transfer Learning and Multi-task Learning Transfer learning [ Pan and Yang, 2009 ] is widely adopted in the NLP community, such as dialogue system [ Mo et al. , 2018 ] and text style transfer [ Fu et al. , 2018 ] .”
A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, methodology): “…or reinforcement learning framework) using transfer learning (Fabbrizio et al., 2004; Forgues et al., 2014; Serban and Pineau, 2015; Serban et al., 2016; Lowe et al., 2015a; Vandyke et al., 2015; Wen et al., 2016; Gašić et al., 2016; Mo et al., 2016; Genevay and Laroche, 2016; Chen et al., 2016)”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “Stylistic conversation generation is closely related to domain adaptation and transfer learning [14, 85, 143, 176].”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, methodology): “[45] used transfer reinforcement learning to eliminate inconsistencies.”
Domain Adaptive Dialog Generation via Meta Learning (2019, background): “Transfer learning (Caruana, 1997a; Bengio, 2012; Cohn et al., 1994; Mo et al., 2018), few-shot learning (Salakhutdinov et al., 2012; Li et al., 2006; Norouzi et al., 2013; Socher et al., 2013) and meta-learning (Finn et al., 2017) are introduced in solving such data scarcity problem in machine…”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “These efforts can also be found in the field of proactive conversation [121, 188, 4].”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Some works pretrained the model in a source domain and expanded the domain area with reinforcement training (Mo et al., 2018; Li et al., 2016d).”
A survey on empathetic dialogue systems (2020, background): “As aforementioned, persona facts and identity features may join forces to personalize the response generation [85, 93, 86, 88], thus makes use of both strengths from unstructured (more context-oriented) and structured data (persona-oriented).”