Efficient Lifelong Learning with A-GEM

By Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, Mohamed Elhoseiny. 2018

1,826 citations recorded by the source corpus.

In lifelong learning, the learner is presented with a sequence of tasks, incrementally building a data-driven prior which may be leveraged to speed up learning of a new task. In this work, we investigate the efficiency of current lifelong approaches, in terms of sample complexity, computational and memory cost. Towards this end, we first introduce a new and a more realistic evaluation protocol, whereby learners observe each example only once and hyper-parameter selection is done on a small and disjoint set of tasks, which is not used for the actual learning experience and evaluation. Second, we introduce a new metric measuring how quickly a learner acquires a new skill. Third, we propose an improved version of GEM (Lopez-Paz & Ranzato, 2017), dubbed Averaged GEM (A-GEM), which enjoys the same or even better performance as GEM, while being almost as computationally and memory efficient as EWC (Kirkpatrick et al., 2016) and other regularization-based methods. Finally, we show that all algorithms including A-GEM can learn even more quickly if they are provided with task descriptors specifying the classification tasks under consideration. Our experiments on several standard lifelong learning benchmarks demonstrate that A-GEM has the best trade-off between accuracy and efficiency.

Subfields: Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Toward Understanding Catastrophic Forgetting in Continual Learning (2019, methodology): “Several algorithms have been developed for utilizing coresets, including gradient episodic memory (GEM) [27], averaged GEM [9], coreset VCL [29], and Stein gradients coreset [10].”
Graph-Based Continual Learning (2020, methodology): “For each dataset, we follow Lopez-Paz and Ranzato (2017); Chaudhry et al. (2018) and adopt the single-pass setting where each task is experienced once and the model only has access to an online stream of data with a batch size of 10.”
Linear Mode Connectivity in Multitask and Continual Learning (2020, methodology): “Among these works, our proposed MC-SGD bares most similarities to rehearsal based methods such us (e.g. Shin et al., 2017; Chaudhry et al., 2018b) and regularization based methods (e.g. Kirkpatrick et al., 2017; Zenke et al., 2017) similar to (Titsias et al., 2019).”
Consistent Representation Learning for Continual Relation Extraction (2022, background): “, 2017), and memory-based methods (Lopez-Paz and Ranzato, 2017; Chaudhry et al., 2018).”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “A-GEM [63] replaces the individual constraints with a global loss of all tasks to improve training efﬁciency.”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, methodology): “ThecodeforASER/AGEM/GDumb: https://github.com/RaptorMai/ online-continual-learning.”
Continual Deep Learning by Functional Regularisation of Memorable Past (2020, background): “Many such memory-based approaches exist (Rebufﬁ et al., 2017; Shin et al., 2017; Aljundi et al., 2019; Chaudhry et al., 2019; Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2018).”
La-MAML: Look-ahead Meta Learning for Continual Learning (2020, methodology): “5 through the k-step MAML update in C-MAML also coincides with optimising the CL objective of AGEM [6]: min θj0 Eτ1:t [ Lt ( Uk(θ j 0) )] = min θj0 t∑ i=1 `i(θj0)− α∂`i ( θj0 ) ∂θj0 · ∂`t ( θj0 ) ∂θj0 .”
Retrospective Adversarial Replay for Continual Learning (2022, methodology): “We also compare against standard baseline methods: (a) Fine-tuning , which trains the model continuously on the stream of tasks without employing any strategy to avoid forgetting, (b) Averaged Gradient Episodic Memory (AGEM) [9] (c) iid-online , which trains a model on the iid-data sampled from enti”
Gradient Regularized Contrastive Learning for Continual Domain Adaptation (2020, methodology): “One representative technique is using episodic memory [26, 29] to store some training samples of old tasks, which are used to overcome catastrophic forgetting via constrained optimization.”
Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism (2022, positive): “Most existing studies on continual learning in visual system focus on continual image classification [3, 7, 12, 24, 35, 42] which aims to alleviate catastrophic forgetting for continually recognizing single object in an image.”
LAMAL: LAnguage Modeling Is All You Need for Lifelong Language Learning (2019, background): “Averaged-GEM (A-GEM) (Chaudhry et al. 2018) is a more efficient version of GEM, and meanwhile, it reaches the same or even better performance than the original GEM.”
Continual Prompt Tuning for Dialog State Tracking (2022, background): “We empirically find that this simple abortion is better than projecting gori onto the normal plane of gref (Chaudhry et al., 2019).”
Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning (2020, methodology): “This is as opposed to reporting only the ﬁnal accuracy—for example [35, 59, 10, 11, 28].”
Efficient Continual Learning with Modular Networks and Task-Driven Priors (2020, methodology): “For instance, Lopez-Paz & Ranzato (2017); Chaudhry et al. (2019b); Rolnick et al. (2019) store past samples in a replay buffer, while Shin et al. (2017) learn to generate new samples from the data distribution of previous tasks and Zhang et al. (2019) computes per-class prototypes.”
Continual learning with direction-constrained optimization (2020, methodology): “In the experiments with Split MNIST and Split CIFAR-100 we use a multi-head setup [2, 9] and we provide task descriptors [28, 18] to the model at both training and testing .”
Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer (2022, methodology): “, [3, 6, 25]) aim to update the model with minimal interference introduced to old tasks.”