Continual Learning in Task-Oriented Dialogue Systems

By Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul A. Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Zhiguang Wang. 2020

148 citations recorded by the source corpus.

Continual learning in task-oriented dialogue systems allows the system to add new domains and functionalities overtime after deployment, without incurring the high cost of retraining the whole system each time. In this paper, we propose a first-ever continual learning benchmark for task-oriented dialogue systems with 37 domains to be learned continuously in both modularized and end-to-end learning settings. In addition, we implement and compare multiple existing continual learning baselines, and we propose a simple yet effective architectural method based on residual adapters. We also suggest that the upper bound performance of continual learning should be equivalent to multitask learning when data from all domain is available at once. Our experiments demonstrate that the proposed architectural method and a simple replay-based strategy perform better, by a large margin, compared to other continuous learning techniques, and only slightly worse than the multitask learning upper bound while being 20X faster in learning new domains. We also report several trade-offs in terms of parameter usage, memory size and training time, which are important in the design of a task-oriented dialogue system. The proposed benchmark is released to promote more research in this direction.

Subfields: Artificial Intelligence, Natural Language Processing, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In LLM Agents & Tool Use

Citations in the corpus

BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, background): “…with appropriate learning systems, could then potentially keep improving over time (Carlson et al., 2010; Kiela et al., 2021; Agichtein et al., 2006; Liu et al., 2021; Madotto et al., 2021; Shuster et al., 2021b), where (Hancock et al., 2019) refer to this approach as a self-feeding chatbot.”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “The applications of continual learning in NLP are also highly diverse, such as dialogue system [130], [131], [260], [260], [282], [298], [437], text classification [87], [172], [325], [460], sentence generation [246], [298], [499], relation learning [343], [357], [455], [465], neural machine transla”
Progressive Prompts: Continual Learning for Language Models (2023, methodology): “For instance, Madotto et al. (2020) proposes AdapterCL which learns a separate adapter block for each task, and Qin & Joty (2021) proposes LFPT5 that learns a large (length 300) soft prompt that is continuously trained on all tasks.”
Continual Training of Language Models for Few-Shot Learning (2022, background): “…(Liu et al., 2019a), translation (Khayral-lah et al., 2018), cross-lingual modeling (Liu et al., 2020b), question answering (Greco et al., 2019) and text classiﬁcation (Ke et al., 2021a,b; Sun et al., 2020; Huang et al., 2021; Chuang et al., 2020; Mehta et al., 2021; Madotto et al., 2020).”
Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, background): “However, when moving to lifelong learning setting, their performance significantly drops (Kirkpatrick et al., 2017; Aljundi et al., 2019; Biesialska et al., 2020; Cui et al., 2021; Ke et al., 2021b; Madotto et al., 2021; Ke et al., 2021a; Feng et al., 2022).”
Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, positive): “In the context of dialogue, a task usually refers to a domain as defined in Madotto et al. (2021) and we will use these two terms interchangeably.”
Continual Sequence Generation with Adaptive Compositional Modules (2022, methodology): “(iv) Adapter+CL (Madotto et al., 2020) inserted adapter (Houlsby et al., 2019) modules into every GPT-2’s layer for each task.”
Continual Prompt Tuning for Dialog State Tracking (2022, methodology): “AdapterCL (Madotto et al., 2021) is the most related to our work, which freezes the pre-trained model and learns an adapter”
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, background): “A dialogue CL dataset is also reported in [38].”
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, methodology): “We use SGD , MWOZ multi-domain dialogue datasets and compare PAGeR with AdapterCL .”
Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation (2022, neutral): “Recent work in this area includes (Mi et al., 2020; Madotto et al., 2020; He et al., 2021; Shin et al., 2017).”
Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022, background): “(3) Architecture-based methods HAT, CTR, and Adapter achieve good performance.”
Continual Pre-training of Language Models (2023, methodology): “The system in (Madotto et al., 2020) learns separate adapters for different domains and thus has no CF or KT. DEMIX (Gururangan et al., 2021) initializes the new adapter with the closest old adapter.”
Dynamic Dialogue Policy Transformer for Continual Reinforcement Learning (2022, background): “Madotto et al. (2021) introduced an architecture called AdapterCL and trained it in a supervised fashion for intent prediction, state tracking, generation and end-to-end learning.”
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey (2024, methodology): “Therefore, it’s natural to leverage PEFT methods for CL tasks [128], [238], [33], [110].”
Drinking From a Firehose: Continual Learning With Web-Scale Natural Language (2020, background): “[73] also constructed a continual learning tasks via re-using”
Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls (2022, background): “…a continual deployment of such a system can then potentially keep improving over time (Carlson et al., 2010; Kiela et al., 2021; Agichtein et al., 2006; Liu and Mazumder, 2021; Madotto et al., 2020; Shus-ter et al., 2020), constituting what Hancock et al. (2019) call a “self-feeding chatbot.””
Fine-tuned Language Models are Continual Learners (2022, methodology): “Mi et al. (2020) and Madotto et al. (2021) perform CL for task oriented dialog systems by using replay based strategy.”