Continual Learning with Tiny Episodic Memories

By Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Kumar Dokania, Philip H. S. Torr, Marc'Aurelio Ranzato. 2019

496 citations recorded by the source corpus.

Learning with less supervision is a major challenge in artificial intelligence. One sensible approach to decrease the amount of supervision is to leverage prior experience and transfer knowledge from tasks seen in the past. However, a necessary condition for a successful transfer is the ability to remember how to perform previous tasks. The Continual Learning (CL) setting, whereby an agent learns from a stream of tasks without seeing any example twice, is an ideal framework to investigate how to accrue such knowledge. In this work, we consider supervised learning tasks and methods that leverage a very small episodic memory for continual learning. Through an extensive empirical analysis across four benchmark datasets adapted to CL, we observe that a very simple baseline, which jointly trains on both examples from the current task as well as examples stored in the memory, outperforms state-of-the-art CL approaches with and without episodic memory. Surprisingly, repeated learning over tiny episodic memories does not harm generalization on past tasks, as joint training on data from subsequent tasks acts like a data dependent regularizer. We discuss and evaluate different approaches to write into the memory. Most notably, reservoir sampling works remarkably well across the board, except when the memory size is extremely small. In this case, writing strategies that guarantee an equal representation of all classes work better. Overall, these methods should be considered as a strong baseline candidate when benchmarking new CL approaches

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

Citations in the corpus

Learnability and Algorithm for Continual Learning (2023, methodology): “The replay methods are: iCaRL (Rebuffi et al., 2017), A-GEM (Chaudhry et al., 2019a), EEIL (Castro et al., 2018), GD (Lee et al., 2019) without external data, DER++ (Buzzega et al., 2020), HAL (Chaudhry et al., 2021), and MORE (Kim et al., 2022a).”
Class Gradient Projection For Continual Learning (2022, methodology): “As for the 5-datasets, similar to [8], we use a reduced ResNet18 architecture.”
Disentangling Transfer in Continual Reinforcement Learning (2022, background): “Continual learning algorithms are often categorized into three classes: regularization-based e.g. [2, 23, 31], parameter isolation e.g. [26] and rehearsal methods e.g. [6, 7]; see also CL survey papers [8, 17, 32].”
Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, methodology): “To evaluate the effectiveness of ORDER for mitigating forgetting in SETS, we compare to SOTA CL baselines, including Experience replay (ER) [15], A-GEM[14], MER [48], GPM [49] and DEGCL [13].”
S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning (2022, methodology): “For instance, memory-based methods alleviate forgetting by either replaying a small set of examples from past tasks saved in a memory [57, 55, 46, 7, 23, 28, 9] or replaying synthesized data from previous tasks using generative models [60, 63].”
Balancing Stability and Plasticity through Advanced Null Space in Continual Learning (2022, methodology): “These method can be roughly divided into four categories: architecture-based methods expand the network or allocate new neurons for new tasks [13,28,42,60]; replayedbased methods interleave old data with current data by storing historical data in a buffer or generating virtual old data [4,8,12,39,40”
Class-incremental Novel Class Discovery (2022, methodology): “Specifically, inspired by the rehearsal-based incremental learning methods [2,7,31] which are known to be effective, we propose to store the base class feature prototypes from the previous task as exemplars, instead of raw images.”
NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks (2022, methodology): “We benchmark NISPA-Replay against four well-known baselines that also rely on replay, namely, Experience Re-play (ER) (Chaudhry et al., 2019b), Dark Experience Replay (DER) (Buzzega et al., 2020), iCaRL (Rebufﬁ et al., 2017), and A-GEM (Chaudhry et al., 2019a).”
Memory Efficient Continual Learning with Transformers (2022, methodology): “[8] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and M Ranzato.”
TRGP: Trust Region Gradient Projection for Continual Learning (2022, methodology): “, (Shin et al., 2017; Chaudhry et al., 2019)) constrain the gradient directions by replaying the data of old tasks during learning of new tasks, in the format of either real data or synthetic data from generative models; and orthogonal-projection based methods (e.”
Representational Continuity for Unsupervised Continual Learning (2021, methodology): “Next, we evaluate the quality of our pre-trained representations using a K-nearest neighbor (KNN) classiﬁer (Wu et al., 2018) To validate knowledge transfer of the learned representations, we adopt the metrics from the SCL literature (Chaudhry et al., 2019b; Mirzadeh et al., 2020).”
Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning (2021, methodology): “BWT(”
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning (2021, methodology): “Experience replay with stored data [2, 3, 7, 8, 13, 17, 22, 27, 38, 44, 45, 46, 57] or samples from a generative model [25, 26, 43, 49, 54] is highly effective when storing training data or training/saving a generative model is possible, unlike the DFCIL setting.”
Optimizing Reusable Knowledge for Continual Learning via Metalearning (2021, methodology): “For our experiments, we use two benchmarks used in previous works [8, 18, 26].”