Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

By Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, Lei Shu. 2021

159 citations recorded by the source corpus.

Continual learning (CL) learns a sequence of tasks incrementally with the goal of achieving two main objectives: overcoming catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. However, most existing techniques focus only on overcoming CF and have no mechanism to encourage KT, and thus do not do well in KT. Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge. Another observation is that most current CL methods do not use pre-trained models, but it has been shown that such models can significantly improve the end task performance. For example, in natural language processing, fine-tuning a BERT-like pre-trained language model is one of the most effective approaches. However, for CL, this approach suffers from serious CF. An interesting question is how to make the best use of pre-trained models for CL. This paper proposes a novel model called CTR to solve these problems. Our experimental results demonstrate the effectiveness of CTR

Subfields: Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “For example, a pre-trained transformer can effectively accommodate downstream tasks by learning only a few task-specific parameters, including adaptor-tuning [165] with inserted fully-connected layers (CTR [208], B-CL [209], CLASSIC [207], CPT [205], CLIF [190], AdapterCL [282], ACM [499], ADA [111]”
Learnability and Algorithm for Continual Learning (2023, background): “Several techniques have also been proposed to carry out knowledge transfer (Ke et al., 2020; 2021; Lin et al., 2022).”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “The other problem for a single memory system is the negative backward transfer [27] in which the performance of the model is decreased due to samples being drawn from entirely different distributions [14].”
Continual Training of Language Models for Few-Shot Learning (2022, background): “…(Liu et al., 2019a), translation (Khayral-lah et al., 2018), cross-lingual modeling (Liu et al., 2020b), question answering (Greco et al., 2019) and text classiﬁcation (Ke et al., 2021a,b; Sun et al., 2020; Huang et al., 2021; Chuang et al., 2020; Mehta et al., 2021; Madotto et al., 2020).”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, background): “Several works have performed knowledge transfer across tasks (Ke, Liu, and Huang 2020; Ke et al. 2021; Schwarz et al. 2018; Fernando et al. 2017; Rusu et al. 2016).”
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning (2023, methodology): “Note that using a pre-trained transformer network and adapter modules is a common practice in existing continual learning methods in the natural language processing community [26].”
Continual Few-shot Intent Detection (2022, background): “However, there are two issues: (i) over-parameterization of PrLMs makes them prone to overfit the current task and cause catastrophic forgetting of previous knowledge (Ke et al., 2021a; Yuan et al., 2021).”
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay (2022, methodology): “Existing continual learning methods in natural language processing commonly use pre-trained transformers and adapter-like modules (Ke et al., 2021).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, methodology): “…another technique leverages large pretrained language models that in practice seem to have sufficient capacity for a massive number of tasks (specifically BERT, Devlin et al., 2019) and adds small modules trained via task-specific masking to learn a sequence of language tasks (Ke et al., 2021).”
A Theory for Knowledge Transfer in Continual Learning (2022, background): “Transferring knowledge in the forward direction has demonstrated some gains (Ke et al., 2021).”
Continual Pre-training of Language Models (2023, methodology): “In NLP, CL has been used for slot ﬁlling (Shen et al., 2019), language learning (Li et al., 2019), sentiment analysis (Ke et al., 2021a), topic modeling (Gupta et al., 2020), question answering (Greco et al., 2019) and text classiﬁcation (Sun et al., 2020; Huang et al., 2021; Chuang et al., 2020).”
Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022, methodology): “We also consider some architecture-based baselines: HAT (Serrà et al., 2018) creates a task-based hard attention during training; CTR (Ke et al., 2021) inserts continual learning plug-ins into BERT to mitigate forgetting and encourage knowledge transfer; Adapter (Madotto et al., 2021) builds…”
AI Autonomy: Self-initiated Open-world Continual Learning and Adaptation (2022, background): “When the tasks are similar, KT is the focus (Ke et al. 2021).”