Continual Lifelong Learning in Natural Language Processing: A Survey

By Magdalena Biesialska, Katarzyna Biesialska, Marta Ruiz Costa-jussà. 2020

272 citations recorded by the source corpus.

Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. However, it is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. Furthermore, CL is particularly challenging for language learning, as natural language is ambiguous: it is discrete, compositional, and its meaning is context-dependent. In this work, we look at the problem of CL through the lens of various NLP tasks. Our survey discusses major challenges in CL and current methods applied in neural network models. We also provide a critical review of the existing CL evaluation methods and datasets in NLP. Finally, we present our outlook on future research directions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

RuleBERT: Teaching Soft Rules to Pre-Trained Language Models (2021, background): “, it started to unlearn what it had learned at pre-fine-tuning (Kirkpatrick et al., 2017; Kemker et al., 2018; Biesialska et al., 2020).”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “In contrast, the latest surveystypicallycollateonlyalocalaspectofcontinuallearning, withrespecttoitsbiologicalunderpinnings[1],[3],[9],[10], specializedsettingsforvisualclassiﬁcation[11],[12],[13],[14], andspeciﬁcextensionsforNLP[15],[16]orRL[17].”
Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, background): “However, when moving to lifelong learning setting, their performance significantly drops (Kirkpatrick et al., 2017; Aljundi et al., 2019; Biesialska et al., 2020; Cui et al., 2021; Ke et al., 2021b; Madotto et al., 2021; Ke et al., 2021a; Feng et al., 2022).”
Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, background): “Despite progress in continual learning (Lange et al., 2019; Parisi et al., 2019; Biesialska et al., 2020; Khetarpal et al., 2020a; Hadsell et al., 2020), there is – to the best of our knowledge – no work that addresses continual reinforcement learning (continual RL) of the dialogue policy, even…”
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 (2021, background): “When it comes to language, earlier approaches to Lifelong Language Learning (LLL) merely focus on a single type of NLP tasks (Wang et al., 2019; d’Autume et al., 2019); see (Biesialska et al., 2020) for a survey.”
Online Coreset Selection for Rehearsal-based Continual Learning (2021, neutral): “In light of this need, continual learning (CL) [40], or lifelong learning, tackles a learning scenario where a model continuously learns over a sequence of tasks [18, 25] within a broad research area, such as classification [15, 9], image generation [47], language learning [24, 4], clinical applicat”
Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021, methodology): “After learning all tasks in the sequence, we seek to minimize the generalization error on all tasks (Biesialska et al., 2020) : We use two commonly-used techniques for this problem setting in our proposed model: • Regularization : in order to preserve knowledge stored in the model, regularization…”
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, background): “…has been a lot of emphasis on continual learning to address these issues for computer vision (Parisi et al., 2018) and natural language processing (Biesialska et al., 2020), wherein the objective is to train a model on a sequence of novel tasks while maintaining performance on prior learned…”
CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability (2021, methodology): “The majority of modern surveys ambiguously conflate these factors as a sequence of tasks (Parisi et al., 2019; Lesort et al., 2019; Hadsell et al., 2020; Lesort et al., 2020; Biesialska et al., 2021).”
Class-Incremental Learning: A Survey (2023, background): “Other surveys either focus on the specific field [20], [21] or lack the performance evolution among state-of-the-arts [8], [22], [23].”
MeLL: Large-scale Extensible User Intent Classification for Dialogue Systems with Meta Lifelong Learning (2021, background): “It is a learning paradigm that continuously accumulates knowledge learned in the past and uses it to help future task learning [2, 33].”
Dynamic Dialogue Policy Transformer for Continual Reinforcement Learning (2022, background): “In the literature, this is referred to as continual learning (Biesialska et al., 2020; Khetarpal et al., 2020; Hadsell et al., 2020).”
TimeLMs: Diachronic Language Models from Twitter (2022, background): “In terms of continual lifelong learning, which is tangential to our main goal, Biesialska et al. (2020) provide a detailed survey on the main techniques proposed in the NLP literature.”
Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, background): “Meta-learning is a general learning strategy that can be used for DG [17, 18, 19, 20] by simulating the meta-train and meta-test tasks in training domains to enhance the performance of DG. Lifelong Learning [21], or continual learning, cares about the learning ability among multiple sequential domai”
Drinking From a Firehose: Continual Learning With Web-Scale Natural Language (2020, unknown): “pared with each other using various modalities in recent continual learning surveys [5], [6], [7].”
Continual Learning and Private Unlearning (2022, methodology): “Due to its generality, CL has been applied to a variety of areas including computer vision (e.g. Kirkpatrick et al., 2017), reinforcement learning (e.g. Kirk-patrick et al., 2017; Riemer et al., 2018), natural language processing (e.g. Biesialska et al., 2020), and robotics (e.g. Liu et al., 2021).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “, 2021), language (Biesialska et al., 2020), and robotics (Lesort et al.”
Towards Exemplar-Free Continual Learning in Vision Transformers: an Account of Attention, Functional and Weight Regularization (2022, background): “This can also be seen in several reviews [4,5,11,29,30,35] reporting few approaches that consider architectures besides CNNs, despite the attempts to investigate CL in RNNs [9,43].”