Episodic Memory in Lifelong Language Learning

By Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama. 2019

341 citations recorded by the source corpus.

We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in this setup. Experiments on text classification and question answering demonstrate the complementary benefits of sparse experience replay and local adaptation to allow the model to continuously learn from new datasets. We also show that the space complexity of the episodic memory module can be reduced significantly (~50-90%) by randomly choosing which examples to store in memory with a minimal decrease in performance. We consider an episodic memory component as a crucial building block of general linguistic intelligence and see our model as a first step in that direction.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “In contrast, Lewis et al. (2020); de Masson d'Autume et al. (2019); Izacard and Grave (2021); Hossain et al. (2020) use unstructured text as knowledge.”
ColdGANs: Taming Language GANs with Cautious Sampling Strategies (2020, background): “In lifelong learning literature [24], it has been shown that 1% of experience replay is sufﬁcient to avoid catastrophic forgetting.”
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering (2021, background): “Extractive readers (Clark and Gardner, 2018, de Masson d’Autume et al., 2019, Wang et al., 2019, Guu et al., 2020, Karpukhin et al., 2020) extract a span from a retrieved document to produce an answer.”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “The applications of continual learning in NLP are also highly diverse, such as dialogue system [130], [131], [260], [260], [282], [298], [437], text classification [87], [172], [325], [460], sentence generation [246], [298], [499], relation learning [343], [357], [455], [465], neural machine transla”
Progressive Prompts: Continual Learning for Language Models (2023, methodology): “For the standard CL benchmark, we use ofﬁcial datasets provided by Zhang et al. (2015) available at http://goo.gl/JyCnZq , following de Masson D’Autume et al. (2019); Zhang et al. (2015).”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, methodology): “Many systems employ this approach, e.g., iCaRL (Rebufﬁ et al. 2017), GEM (Lopez-Paz and Ranzato 2017), A-GEM (Chaudhry et al. 2019), RPSnet (Rajasegaran et al. 2019) and others (Rusu et al. 2016; Wu et al. 2019; Rolnick et al. 2019b; de Masson d’Autume et al. 2019; Hou et al. 2019).”
Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, background): “Though experience replay (Lopez-Paz and Ranzato, 2017; de Masson d’Autume et al., 2019; Guo et al., 2020; Han et al., 2020; Zhao et al., 2022) and knowledge distillation (Chuang et al., 2020; Cao et al., 2020) have shown to be effective in overcoming catastrophic forgetting, they highly rely on the…”
ELLE: Efficient Lifelong Pre-training for Emerging Data (2022, methodology): “…this goal, including (1) memory-based methods (Rebufﬁ et al., 2017; Rolnick et al., 2019), which perform experience replay with authentic data (de Masson d’Autume et al., 2019), automatically generated data (Sun et al., 2020), or previously computed gradients (Lopez-Paz and Ranzato, 2017)…”
Memory Efficient Continual Learning with Transformers (2022, background): “[11] propose to use such a memory module for sparse experience replay and local adaptation in the language domain.”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, background): “On an existing benchmark spanning 5 diverse NLP tasks (de Masson d’Autume et al., 2019), we observe that increasing the capacity of the model and diversity of the pre-training corpus play an important role in alleviating forgetting.”
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 (2021, methodology): “When it comes to language, earlier approaches to Lifelong Language Learning (LLL) merely focus on a single type of NLP tasks (Wang et al., 2019; d’Autume et al., 2019); see (Biesialska et al.”
Towards Continual Knowledge Learning of Language Models (2021, methodology): “One might consider implementing Continual Learning (CL) methods (Sun et al., 2020; d’Autume et al., 2019) to address this problem.”
Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, positive): “Akin to our work, Gdumb [68] and MBPA++ [14] also train the model at inference time.”
Continual Learning by Using Information of Each Class Holistically (2021, methodology): “Representative work includes GEM (Lopez-Paz and Ranzato 2017), A-GEM (Chaudhry et al. 2019), and many others (Rusu et al. 2016; Rebufﬁ, Kolesnikov, and Lampert 2017; Wu et al. 2019; Rolnick et al. 2019; de Masson d’Autume et al. 2019; Hou et al. 2019).”
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning (2021, methodology): “…that look to learn from sequentially arriving tasks, known as continual learning (CL) or lifelong learning (Robins, 1995; Sun et al., 2020; de Masson d’Autume et al., 2019), mainly focus on retaining the performance on seen tasks when the model is continuously updated on new tasks (i.e., to…”
Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021, methodology): “Continual Learning Existing continual learning research can be broadly divided into four categories: (i) replay-based method, which remind models of information from seen tasks via experience replay (d’Autume et al., 2019), distillation (Rebuffi et al.”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “While there are major differences between the nature of activity in artificial and spiking networks, machine learning–inspired replay methods could still provide insights into neural activity selection during sleep replay.”