Continual Learning for Natural Language Generation in Task-oriented Dialog Systems

By Fei Mi, Liangwei Chen, Mengjie Zhao, Minlie Huang, Boi Faltings. 2020

84 citations recorded by the source corpus.

Natural language generation (NLG) is an essential component of task-oriented dialog systems. Despite the recent success of neural approaches for NLG, they are typically developed in an offline manner for particular domains. To better fit real-life applications where new data come in a stream, we study NLG in a “continual learning” setting to expand its knowledge to new domains or functionalities incrementally. The major challenge towards this goal is catastrophic forgetting, meaning that a continually trained model tends to forget the knowledge it has learned before. To this end, we propose a method called ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying prioritized historical exemplars, together with an adaptive regularization technique based on Elastic Weight Consolidation. Extensive experiments to continually learn new domains and intents are conducted on MultiWoZ-2.0 to benchmark ARPER with a wide range of techniques. Empirical results demonstrate that ARPER significantly outperforms other methods by effectively mitigating the detrimental catastrophic forgetting issue.

Subfields: Artificial Intelligence, Dialogue & Conversational AI, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Dialogue & Conversational AI

In Meta & Continual Learning

Citations in the corpus

Modifying Memories in Transformer Models (2020, background): “Continual learning, with recent extensions to language models (Sun et al., 2020; Liu et al., 2019; Mi et al., 2020; Chuang et al., 2020), aims to learn a new task while preserving the performance on the previous tasks without access to their data.”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “MBPA++ [442], EMAR [465], DnR [396], ARPER [298], Total Recall [253]), generative replay (PAGeR [415], LAMOL [395], ACM [499], NER [434]), parameter allocation (TPEM [131]), modular network (ProgModel [377]), meta-learning (MetaMBPA++ [442], MeLL [422], CML [455]), etc.”
Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, background): “Other works concentrated on dialogue state tracking (Wu et al., 2019) or natural language generation (Mi et al., 2020; Geng et al., 2021).”
Continual Sequence Generation with Adaptive Compositional Modules (2022, methodology): “The ﬁrst one continually learns new tasks on old parameters (Fig 1 a), with approaches like experience replay (Sun et al., 2019; Chuang et al., 2020) and regularization (Mi et al., 2020) to maintain old knowledge.”
Continual Prompt Tuning for Dialog State Tracking (2022, methodology): “For continual learning in dialog system, variants of general CL methods have been applied (Lee, 2017; Shen et al., 2019; Wu et al., 2019; Mi et al., 2020; Geng et al., 2021).”
Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021, methodology): “ARPER (Mi et al., 2020) adds an adaptive EWC regularization on the EMR loss, where the memory instances are sampled with a unique sampling method called PRIOR.”
Co-Transport for Class-Incremental Learning (2021, background): “, e-commerce product search [43], video action recognition[54], natural language generation [30], multimedia retrieval [40], and social media topic mining [49].”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “Moreover, multiple works are proposed to address general CL language learning [138, 139] and specific language tasks, such as dialogue systems [140, 141, 142], image captioning [143], sentiment classification [144] and sentence representation learning [145].”
Continual Learning in Task-Oriented Dialogue Systems (2020, background): “Therefore the assumption of minimizing the number of samples in memory is valid and widely used in the CL literature (Mi et al., 2020).”
Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation (2022, methodology): “In (Mi et al., 2020), using 250 exemplars for computing Fisher information matrix boosted the performances to a large degree.”
Dynamic Dialogue Policy Transformer for Continual Reinforcement Learning (2022, background): “, 2020), the task-oriented dialogue systems research community has barely touched the topic (Lee, 2017; Wu et al., 2019; Mi et al., 2020; Madotto et al., 2021; Geng et al., 2021).”
Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022, background): “One line of studies (Rebufﬁ et al., 2017; Chaudhry et al., 2019; Lopez-Paz and Ranzato, 2017; Mi et al., 2020; Han et al., 2020; Liu et al., 2021b) keeps a small number of real samples from old tasks for replaying.”
Continual-T0: Progressively Instructing 50+ Tasks to Language Models Without Forgetting (2022, background): “Mi et al. (2020) and Madotto et al. (2021) Most of the aforementioned works fall into the 2 scenarios differentiated by Lomonaco and Maltoni (2017): 1) learning new data of known classes (on-line learning), and 2) learning new classes (class-incremental learning).”
Fine-tuned Language Models are Continual Learners (2022, methodology): “Mi et al. (2020) and Madotto et al. (2021) perform CL for task oriented dialog systems by using replay based strategy.”
CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems (2021, methodology): “GPT2 is applied by Budzianowski and Vulic (2019); Mi et al. (2020) to train a response generation model.”