Continual Learning with Deep Generative Replay

By Hanul Shin, Jung Kwon Lee, Jaehong Kim, Jiwon Kim. Neural Information Processing Systems, 2017

2,523 citations recorded by the source corpus.

Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting. Although simply replaying all previous data alleviates the problem, it requires large memory and even worse, often infeasible in real world applications where the access to past data is limited. Inspired by the generative nature of hippocampus as a short-term memory system in primate brain, we propose the Deep Generative Replay, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"). With only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task. We test our methods in several sequential learning settings involving image classification tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

View Paper: https://arxiv.org/abs/1705.08690

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “In recent years, interest in continual learning has risen [1, 21, 26, 27, 29, 39, 40, 42, 51], especially from the deep learning research community, due to its potential to reduce training time and training set sizes (e.g., by continuously adapting from previous models), both of which are critical…”
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines (2018, methodology): “For comparison, we pick several popular methods (EWC[3], online EWC[23], SI[9], MAS[24], LwF[25]) and state-of-the-art rehearsal-based methods (DGR[8], RtF[14]) with generative model.”
Born Again Neural Networks (2018, background): “A number of recent papers (Furlanello et al., 2016; Li & Hoiem, 2016; Shin et al., 2017) employ KD for the purpose of minimizing forgetting in continual learning.”
Experience Replay for Continual Learning (2018, background): “Research in this vein has included prioritized replay for maximizing the impact of rare experiences [28], learning from human demonstration data seeded into a buffer [9], and methods for approximating replay buffers with generative models [30].”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “DGR [160] provides an initial framework in which learning each generation task is accompanied with replaying generated data sampled from the old generative model, so as to inherit the previously-learned knowledge.”
DisCoRL: Continual Reinforcement Learning via Policy Distillation (2019, methodology): “The fourth method is generative replay [Shin et al., 2017, Lesort et al., 2018, Wu et al., 2018, Lesort et al., 2018], where a generative model is used as a memory to produce samples from previous tasks.”
Deep Generative Dual Memory Network for Continual Learning (2017, methodology): “We point out that a similar idea has also been proposed recently by Shin et al. (2017) independently, where they use a generative model to sample previously seen inputs and a feedforward network (solver) to learn task labels.”
Policy Consolidation for Continual Reinforcement Learning (2019, methodology): “…ones that allocate more neural resources over time (Ring, 1997; Rusu et al., 2016), and ones that explicitly store (or train generative models to mimic) past data which are used in various ways to improve the memory of the network (Isele & Cosgun, 2018; Sprechmann et al., 2018; Shin et al., 2017).”
Progress & Compress: A scalable framework for continual learning (2018, background): “A large body of literature recognises the importance of the continual learning problem, and there has been some increased interest in the topic recently (e.g. Rusu et al., 2016a; Shin et al., 2017; Lopez-Paz et al., 2017; Nguyen et al., 2017; Kirkpatrick et al., 2017; Chaudhry et al., 2018).”
Continual Reinforcement Learning with Complex Synapses (2018, background): “, 2017) and building generative models to mimic old datasets that are no longer available (Shin et al., 2017), but these are all orthogonal to the approach used in this paper and could be combined with it.”
Learnability and Algorithm for Continual Learning (2023, methodology): “Pseudoreplay methods generate pseudo replay data for old tasks to serve as the replay data (Kamra et al., 2017; Shin et al., 2017; Wu et al., 2018; Seff et al., 2017; Kemker & Kanan, 2018; Hu et al., 2019; Rostami et al., 2019; Ostapenko et al., 2019).”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “, 2021; Heckel, 2022), memory-based methods (Robins, 1993; Shin et al., 2017; Rebuffi et al., 2017; Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2019; Aljundi et al., 2019b; Buzzega et al., 2020; Verwimp et al., 2021; Bang et al., 2021; Kim et al., 2022; Zhang et al., 2022; Wang et al., 2022a; Saha &”
Towards Label-Efficient Incremental Learning: A Survey (2023, background): “However, thanks to efforts in regularizing neural network weights from abruptly drifting [ Li and Hoiem, 2017; Zenke et al. , 2017 ] , or replaying previous data from the memory [ Shin et al. , 2017 ] , forgetfulness has reduced dramatically in years.”
RMM: Reinforced Memory Management for Class-Incremental Learning (2023, background): “…are either task-incremental learning—each phase corresponds to a task (dataset) that contains new data of all seen classes [7, 10, 19, 29, 43, 47, 57], or class-incremental learning (CIL)—each phase contains data of a new set of classes, i.e., classes are unseen [4, 6, 18, 25, 31, 37, 40,…”
A soft nearest-neighbor framework for continual semi-supervised learning (2022, neutral): “The first category comprises regularization-based methods [13,14,20,22,24,25,30,45,54], which address the problem of catastrophic forgetting by introducing appropriate regularization terms in the objective function or identifying a set of parameters that are most relevant for certain tasks.”