Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models

By Yi Luan, Chris Brockett, William B. Dolan, Jianfeng Gao, Michel Galley. 2017

81 citations recorded by the source corpus.

Building a persona-based conversation agent is challenging owing to the lack of large amounts of speaker-specific conversation data for model training. This paper addresses the problem by proposing a multi-task learning approach to training neural conversation models that leverages both conversation data across speakers and other types of data pertaining to the speaker and speaker roles to be modeled. Experiments show that our approach leads to significant improvements over baseline model quality, generating responses that capture more precisely speakers’ traits and speaking styles. The model offers the benefits of being algorithmically simple and easy to implement, and not relying on large quantities of data representing specific individual speakers.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Learning from Dialogue after Deployment: Feed Yourself, Chatbot! (2019, background): “Since the question of how to incorporate external knowledge (such as profiles) in dialogue is an open research question of its own (Li et al., 2016; Luan et al., 2017; Luo et al., 2018) and we are primarily interested in the question of learning from dialogue, we discard the profiles and simply trai”
Revealing Persona Biases in Dialogue Systems (2021, neutral): “, age, gender, dialect, other personal information) (Li et al. 2016; Luan et al. 2017).”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “[81] proposed a multi-task learning approach where the response generation and utterance representation are treated as two sub-tasks for speaker role adaptation.”
Learning Personalized End-to-End Goal-Oriented Dialog (2018, background): “Instead of assigning personality to agents (Li et al. 2016; Luan et al. 2017; Qian et al. 2017), our model pays more attention to the user persona and aims to make agents more adaptive to different kinds of interlocutors.”
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (2018, methodology): “We compare our model with the following base-lines on S CI ERCdataset: • LSTM+CRF The state-of-the-art NER sys-tem (Lample et al., 2016), which applies CRF on top of LSTM for named entity tagging, the approach has also been used in scientiﬁc term extraction (Luan et al., 2017b).”
Jointly Optimizing Diversity and Relevance in Neural Response Generation (2019, methodology): “In contrast to previous multi-task conversation model (Luan et al., 2017), where S2S and AE are trained alternately, our approach trains S2S and AE at the same time by minimizing the loss function of Equation 3.”
Microsoft Icecaps: An Open-Source Toolkit for Conversation Modeling (2019, background): “By unifying a conversational sequence-to-sequence model and an autoencoder with a shared decoder, multi-task learning can personalize the conversational model (Luan et al., 2017).”
Polite Dialogue Generation Without Parallel Data (2018, methodology): “In order to obtain a persona-based conversational agent, Luan et al. (2017) proposed a multi-task learning (MTL) based approach: they train a Seq2seq model with conversation data and an au-toencoder with non-conversational persona-related data from target speakers, and share the decoder parameters…”
A general framework for information extraction using dynamic span graphs (2019, methodology): “…explored joint modeling (Miwa and Bansal, 2016; Zhang et al., 2017; Singh et al., 2013; Yang and Mitchell, 2016)) and multi-task learning (Peng and Dredze, 2015; Peng et al., 2017; Luan et al., 2018a, 2017a) as methods to share representational strength across related information extraction tasks.”
Knowledge-Grounded Dialogue Generation with Term-level De-noising (2021, background): “Luan et al. (2017) expanded the scope and introduced personality information into the model.”
Neural personalized response generation as domain adaptation (2017, methodology): “[20] took the seq2seq model and autoencoder model for response generation as two tasks and proposed a multi-task learning framework for speaker role adaptation.”
Diversifying Dialogue Generation with Non-Conversational Text (2020, background): “Luan et al. (2017) extended it to conversational models for speaker-role adaptation.”