Scalable Recollections for Continual Lifelong Learning

By Matthew Riemer, Tim Klinger, Djallel Bouneffouf, Michele M. Franceschini. 2017

76 citations recorded by the source corpus.

Given the recent success of Deep Learning applied to a variety of single tasks, it is natural to consider more human-realistic settings. Perhaps the most difficult of these settings is that of continual lifelong learning, where the model must learn online over a continuous stream of non-stationary data. A successful continual lifelong learning system must have three key capabilities: it must learn and adapt over time, it must not forget what it has learned, and it must be efficient in both training time and memory. Recent techniques have focused their efforts primarily on the first two capabilities while questions of efficiency remain largely unexplored. In this paper, we consider the problem of efficient and effective storage of experiences over very large time-frames. In particular we consider the case where typical experiences are O(n) bits and memories are limited to O(k) bits for k

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “In contrast, autoencoder-based strategies, such as FearNet [165], SRM [169] and EEC [168], can explicitly control the labels of the generated data, although relatively blurred.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “efficient, though now only consisting of approximate recollections (Riemer et al., 2017; Caccia et al., 2019).”
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020, background): “2015b), RALSTM (Tran and Nguyen, 2017), SCVAE (Tseng et al., 2018). Recent works have considered the domain adaptation setting. Tseng et al. (2018); Tran and Nguyen (2018b) proposed to learn domain-invariant representations using VAE (Kingma and Welling, 2013).”
Compacting, Picking and Growing for Unforgetting Continual Learning (2019, methodology): “Data-preserving approaches (such as [32, 3, 11, 34]) are designed to directly save important data or latent codes as an efficient form, while memory-replay approaches [41, 46, 13, 45, 27] introduce additional memory models such as GANs for keeping data information or distribution in an indirect way.”
Online Continual Learning with Maximally Interfered Retrieval (2019, methodology): “We note that this is not the case for the discrete autoencoder proposed in [31].”
Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference (2018, methodology): “This procedure closely follows recent enhancements discussed in (Zhang & Sutton, 2017; Riemer et al., 2017b;a) The sample function randomly samples k − 1 examples from the memory buffer M and interleaves them with the current example to form a single size k batch.”
Generative replay with feedback connections as a general strategy for continual learning (2018, background): “The promising idea of storing hidden representations, which besides reducing memory storage demands could also address the privacy issue, is further worked out in [33].”
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification (2020, background): “Riemer et al. [76] use the memory to store discrete latent embeddings from a Variational Autoencoder that allows generation of previous task data for training.”
Rethinking Experience Replay: a Bag of Tricks for Continual Learning (2020, methodology): “This is corroborated by a line of very recent works [14], [19], [15], [16], [25], [26], [27] that argue for its superior effectiveness in comparison to other methods and propose various extensions to it.”
Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors (2021, background): “The basic idea is to train an auto-encoder [26], [27], [28] or generative adversarial network”
Building a Subspace of Policies for Scalable Continual Learning (2022, methodology): “Other methods maintain a buffer of experience from prior tasks to alleviate forgetting [24, 18, 34, 36, 7].”
Effects of Auxiliary Knowledge on Continual Learning (2022, methodology): “Most notably, rehearsal-based approaches do so by maintaining a working memory of previously seen examples and interleaving them with the input data [3], [10], [28], [29], [30], while pseudorehearsal methods approximate this procedure with a generative model [31], [11].”
On the Role of Weight Sharing During Deep Option Learning (2019, background): “Finally, to provide additional clarity, in the ﬁnal row of Figure 8 we follow (Riemer et al., 2019a) and try to get a sense of the transfer and interference behavior between gradients over the course of learning.”
Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2 (2022, background): “Others have explored sophisticated strategies for experience replay in DQN (e.g. Isele & Cosgun, 2018; Riemer et al., 2019; Wang et al., 2021; Caccia et al., 2020; Zhang & Sutton, 2017; Schaul et al., 2015) in lifelong RL, but these are not generative in nature.”
Forget Me Not: Reducing Catastrophic Forgetting for Domain Adaptation in Reading Comprehension (2019, methodology): “Several studies present methods to reduce forgetting with limited or no access to previous data [9], [10], [13], [21],”
Self-Net: Lifelong Learning via Continual Self-Modeling (2018, methodology): “Some methods speciﬁcally use autoencoders for this task (Zhou et al., 2012; Riemer et al., 2017; Parisi et al., 2018b).”