Toward Continual Learning for Conversational Agents

By Sungjin Lee. 2017

41 citations recorded by the source corpus.

While end-to-end neural conversation models have led to promising advances in reducing hand-crafted features and errors induced by the traditional complex system architecture, they typically require an enormous amount of data. Previous studies adopted a hybrid approach with knowledge-based components to abstract out domain-specific things or to augment data to cover more diverse patterns. On the contrary, we propose to directly address the problem using the recent development in the space of continual learning for neural models. Specifically, we adopt a domain-independent neural conversational model and introduce a novel neural continual learning algorithm that allows the conversational agent to accumulate skills across different tasks in a data-efficient way. To the best of our knowledge, this is the first work that applies continual learning for conversation systems. We verified the efficacy of our method through a conversational skill transfer from synthetic dialogs or human-human dialogs to human-computer conversations in a customer support domain.

Subfields: Artificial Intelligence, Dialogue & Conversational AI, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Dialogue & Conversational AI

In Meta & Continual Learning

Citations in the corpus

Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, background): “Works that allow new actions to be added to the action set compare the encoded state and action embeddings with each other (Lee, 2017; Xu et al., 2020; Vlasov et al., 2019), suggesting that exploiting similarities is key not only for state representations but also for action prediction.”
ConTinTin: Continual Learning from Task Instructions (2022, background): “Typical work in NLP includes sentiment classiﬁcation (Chen et al., 2015; Xia et al., 2017), conversational agents (Lee, 2017), text classiﬁcation, and question answering (d’Autume et al., 2019), etc. TCL tries to learn distinct tasks sequentially.”
Continual Prompt Tuning for Dialog State Tracking (2022, methodology): “For continual learning in dialog system, variants of general CL methods have been applied (Lee, 2017; Shen et al., 2019; Wu et al., 2019; Mi et al., 2020; Geng et al., 2021).”
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020, background): “Lee (2017) ﬁrstly studied the continual learning setting for dialog state tracking in task-oriented dialog systems.”
LAMOL: LAnguage MOdeling for Lifelong Language Learning (2019, background): “, 2017), conversational agents (Lee, 2017), word representation learning (Xu et al.”
Continual Learning for Sentence Representations Using Conceptors (2019, background): “However, in open-domain NLP systems such as conversational agents, oftentimes we are facing a dynamic environment, where training data are accumulated sequentially over time and the distributions of training data vary with respect to external input (Lee, 2017; Mathur and Singh, 2018).”
Lifelong Language Knowledge Distillation (2020, background): “…been studied for lifelong learning in the previous work, including sentiment analysis (Chen et al., 2015; Xia et al., 2017), conversational agents (Lee, 2017), word and sentence representation learning (Xu et al., 2018; Liu et al., 2019), text classiﬁcation, and question answering (d’Autume et…”
Defining Benchmarks for Continual Few-Shot Learning (2020, methodology): “Other regularization techniques have been recently presented which follow a similar approach (Zenke et al., 2017; Lee, 2017; He & Jaeger, 2018).”
The Rapidly Changing Landscape of Conversational Agents (2018, background): “However Lee et al in [22] take a different approach to add knowledge to conversational agents.”
Continual learning in recurrent neural networks (2020, methodology): “nke et al., 2017). Variants of SI have been used for different sequential datasets, but have not been systematically compared against other established methods (Yang et al., 2019; Masse et al., 2018; Lee, 2017). Fixed expansion layers (Coop and Arel, 2012) are another method to limit the plasticity ”
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey (2024, background): “2) Continual Learning (CL): CL aims to learn a sequence of new tasks over time within one single model, which has broad application in scenarios such as dialogue systems [100], information extraction systems [14], and question answering systems [201].”
LAMAL: LAnguage Modeling Is All You Need for Lifelong Language Learning (2019, methodology): “The tasks have been studied including sentiment analysis (Chen, Ma, and Liu 2015; Xia, Jiang, and He 2017), conversational agent (Lee 2017), word representation learning (Xu et al. 2018), sentence representation learning (Liu, Ungar, and Sedoc 2019), text classification, and question answering…”
Continual Learning in Recurrent Neural Networks with Hypernetworks (2020, methodology): “Variants of SI have been used for different sequential datasets, but have not been systematically compared against other established methods [21, 7, 22].”
Dynamic Dialogue Policy Transformer for Continual Reinforcement Learning (2022, background): “Works that allow new actions to be added to the action set compare the encoded state and action embeddings with each other (Lee, 2017; Xu et al., 2020; Vlasov et al., 2019), suggesting that exploiting similarities is key not only for state representations but also for action prediction.”