Gradient Episodic Memory for Continual Learning
By David Lopez-Paz, Marc'Aurelio Ranzato. 2017
3,589 citations recorded by the source corpus.
One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we study the problem of continual learning, where the model observes, once and one by one, examples concerning a sequence of tasks. First, we propose a set of metrics to evaluate models learning over a continuum of data. These metrics characterize models not only by their test accuracy, but also in terms of their ability to transfer knowledge across tasks. Second, we propose a model for continual learning, called Gradient Episodic Memory (GEM) that alleviates forgetting, while allowing beneficial transfer of knowledge to previous tasks. Our experiments on variants of the MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when compared to the state-of-the-art.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Sergey Levine
- Jianfeng Gao
- Dacheng Tao
- Quoc V. Le
- Percy Liang
- Trevor Darrell
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Ming-Hsuan Yang
- Bernt Schiele
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Shimon Whiteson
- Joelle Pineau
- Yuxi Li
- Abhishek Gupta
- Ofir Nachum
Citations in the corpus
- Transfer Learning in Natural Language Processing (2019, background): “As part of this overview, we will also highlight connections to other related and promising directions of research such as meta-learning (Gu et al., 2018), multilingual transfer learning, and continual learning (Lopez-Paz and Ranzato, 2017).”
- Toward Understanding Catastrophic Forgetting in Continual Learning (2019, methodology): “Several algorithms have been developed for utilizing coresets, including gradient episodic memory (GEM) [27], averaged GEM [9], coreset VCL [29], and Stein gradients coreset [10].”
- Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines (2018, background): “This evaluation scenario is widely adopted [3, 4, 5, 6, 7, 8, 9], despite criticism that it is less challenging in terms of forgetting [10].”
- Continual Lifelong Learning with Neural Networks: A Review (2018, background): “Maltoni and Lomonaco (2018) proposed the AR1 model for single-incremental-task scenarios which combines architectural and regularization strategies.”
- Experience Replay for Continual Learning (2018, background): “In recent years, however, there has been renewed interest in overcoming catastrophic forgetting in RL and in supervised learning from streaming data, where it is not possible simply to reorder the data (see e.g. [8, 17, 24, 26]).”
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “Overall performance is typically evaluated by average accuracy (AA) [44], [45] and average incremental accuracy (AIA) [46], [47].”
- Memory Efficient Experience Replay for Streaming Learning (2018, methodology): “Streaming classification is distinct from incremental batch learning, which has recently received much attention [40], [41], [48], [53], [64], [73], and it has unique challenges.”
- Continual Learning via Sequential Function-Space Variational Inference (2023, methodology): “As in related work (Lopez-Paz and Ranzato, 2017; Pan et al., 2020), we compute the forward transfer (FT) and backward transfer (BT) for S - FSVI on split CIFAR .”
- Incorporating neuro-inspired adaptability for continual learning in artificial intelligence (2023, methodology): “We mainly consider three evaluation metrics for visual classification tasks, such as average accuracy (AAC), forward transfer (FWT) and backward transfer (BWT) [6, 75]: where A t,i is the test accuracy of task i after continual learning of task t , and ˜ a i is the test accuracy of each task i learn”
- Learnability and Algorithm for Continual Learning (2023, neutral): “This is also referred as backward transfer in other literature (Lopez-Paz & Ranzato, 2017).”
- The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “…Park et al., 2019; Yin et al., 2020; Zhou et al., 2021; Heckel, 2022), memory-based methods (Robins, 1993; Shin et al., 2017; Rebuffi et al., 2017; Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2019; Aljundi et al., 2019b; Buzzega et al., 2020; Verwimp et al., 2021; Bang et al., 2021; Kim et al.,…”