Deep Generative Dual Memory Network for Continual Learning

By Nitin Kamra, Umang Gupta, Yan Liu. 2017

162 citations recorded by the source corpus.

Despite advances in deep learning, artificial neural networks do not learn the same way as humans do. Today, neural networks can learn multiple tasks when trained on them jointly, but cannot maintain performance on learnt tasks when tasks are presented one at a time -- this phenomenon called catastrophic forgetting is a fundamental challenge to overcome before neural networks can learn continually from incoming data. In this work, we derive inspiration from human memory to develop an architecture capable of learning continuously from sequentially incoming tasks, while averting catastrophic forgetting. Specifically, our model consists of a dual memory architecture to emulate the complementary learning systems (hippocampus and the neocortex) in the human brain, and maintains a consolidated long-term memory via generative replay of past experiences. We (i) substantiate our claim that replay should be generative, (ii) show the benefits of generative replay and dual memory via experiments, and (iii) demonstrate improved performance retention even for small models with low capacity. Our architecture displays many important characteristics of the human memory and provides insights on the connection between sleep and learning in humans.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Computer Vision and Pattern Recognition, Reinforcement Learning, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Generative Models

Citations in the corpus

Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “Other algorithmic ideas to prevent catastrophic forgetting include moment matching [25], learning without forgetting [26], and deep generative replay [13, 19, 42].”
Continual Lifelong Learning with Neural Networks: A Review (2018, background): “ing sleep phases. To do this, FearNet’s PFC model is a generative neural network that creates pseudosamples that are then intermixed with recently observed examples stored in its hippocampal network. Kamra et al. (2018) presented a similar dual-memory framework for continual learning that also uses ”
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “Kamra et al. (2018) presented a similar dual-memory framework for lifelong learning that uses a variational autoencoder as a generative model for pseudo-rehearsal.”
Learnability and Algorithm for Continual Learning (2023, methodology): “Pseudoreplay methods generate pseudo replay data for old tasks to serve as the replay data (Kamra et al., 2017; Shin et al., 2017; Wu et al., 2018; Seff et al., 2017; Kemker & Kanan, 2018; Hu et al., 2019; Rostami et al., 2019; Ostapenko et al., 2019).”
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, background): “Rehearsal with stored data [3–5, 7, 9, 10, 18, 19, 24, 29, 41, 50–52, 63, 67] or samples from a generative model [27, 28, 47, 55, 59] is highly effective when storing training data or training/saving a generative model is possible.”
A Theoretical Study on Solving Continual Learning (2022, background): “Some systems learn to generate pseudo training data of old tasks to be used in replay, called pseudo-replay [41, 42, 43, 44, 45, 46, 21, 47, 48].”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, background): “Related work includes (Shin et al. 2017; Wu et al. 2018; Kamra, Gupta, and Liu 2017; Seff et al. 2017; Hu et al. 2019; Ostapenko et al. 2019; Hayes et al. 2019; von Oswald et al. 2020).”
Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System (2022, methodology): “Kamra et al. (2017) utilizes two generative models in a dual memory architecture.”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “Some works built generators for previous tasks so that learning is done using a mixed set of real data of the new task and generated data of previous tasks (called pseudo-replay ) (Shin et al., 2017; Kamra et al., 2017; Rostami et al., 2019; Hu et al., 2019).”
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, methodology): “The methods in [54, 20, 47, 14] learn data generators and generate old task data for learning a new task.”
RECALL: Replay-based Continual Learning in Semantic Segmentation (2021, background): “Generative models are usually GANs [39, 43, 17] or auto-encoders [21].”
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning (2021, methodology): “An intuitive approach for DFCIL is to simultaneously train a generative model to be sampled for replay [25, 26, 49, 58].”
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, methodology): “The methods in (Shin et al., 2017; Kamra et al., 2017; Rostami et al., 2019; He and Jaeger, 2018) build data generators for previous tasks so that in learning the new task, they can use some generated data for previous tasks to help avoid forgetting.”
Continual Learning by Using Information of Each Class Holistically (2021, methodology): “Some methods also learn to generate pseudo old data and use them to jointly train with the new task data (Shin et al. 2017; Wu et al. 2018; Kamra, Gupta, and Liu 2017; Seff et al. 2017; Hu et al. 2019; Lesort et al. 2018; Ostapenko et al. 2019; Hayes et al. 2019) (also known as rehearsal (von…”
Continual Learning with Fully Probabilistic Models (2021, methodology): “Typical models used as generators are Generative Adversarial Networks (GANs), Variational AutoEncoders (VAEs) and their variants, see [25] and [26].”
Rectification-based Knowledge Retention for Continual Learning (2021, methodology): “The methods proposed in [11, 10] use a custom architecture to produce pseudo samples for the older tasks to be used for rehearsal.”
Gradient Projection Memory for Continual Learning (2021, methodology): “Results are listed in Table 11, where the baselines are reported from DGDMN.”
Distill and Replay for Continual Language Learning (2020, methodology): “One is data-based method that achieves continual learning through reproducing the data distribution of old tasks (Kamra et al., 2017; Chaudhry et al., 2018).”
Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting (2020, methodology): “In contrast, pseudo-rehearsal methods generate the replay samples using a generative model such as an autoencoder (Kemker & Kanan, 2017) or a GAN (Kamra et al., 2017; Shin et al., 2017).”