Closed-Loop GAN for continual Learning

By Amanda Rios, Laurent Itti. 2018

21 citations recorded by the source corpus.

Sequential learning of tasks using gradient descent leads to an unremitting decline in the accuracy of tasks for which training data is no longer available, termed catastrophic forgetting. Generative models have been explored as a means to approximate the distribution of old tasks and bypass storage of real data. Here we propose a cumulative closed-loop generator and embedded classifier using an AC-GAN architecture provided with external regularization by a small buffer. We evaluate incremental learning using a notoriously hard paradigm, single headed learning, in which each task is a disjoint subset of classes in the overall dataset, and performance is evaluated on all previous classes. First, we show that the variability contained in a small percentage of a dataset (memory buffer) accounts for a significant portion of the reported accuracy, both in multi-task and continual learning settings. Second, we show that using a generator to continuously output new images while training provides an up-sampling of the buffer, which prevents catastrophic forgetting and yields superior performance when compared to a fixed buffer. We achieve an average accuracy for all classes of 92.26% in MNIST and 76.15% in FASHION-MNIST after 5 tasks using GAN sampling with a buffer of only 0.17% of the entire dataset size. We compare to a network with regularization (EWC) which shows a deteriorated average performance of 29.19% (MNIST) and 26.5% (FASHION). The baseline of no regularization (plain gradient descent) performs at 99.84% (MNIST) and 99.79% (FASHION) for the last task, but below 3% for all previous tasks. Our method has very low long-term memory cost, the buffer, as well as negligible intermediate memory storage.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition, Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

In Generative Models

In Meta & Continual Learning

Citations in the corpus

Experience Replay for Continual Learning (2018, background): “In recent years, however, there has been renewed interest in overcoming catastrophic forgetting in RL and in supervised learning from streaming data, where it is not possible simply to reorder the data (see e.g. [8, 17, 24, 26]).”
Linear Mode Connectivity in Multitask and Continual Learning (2020, background): “Experience replay: Experience replay methods build and store a memory of the knowledge learned so far (Rebuffi et al., 2016; Lopez-Paz & Ranzato, 2017; Shin et al., 2017; Riemer et al., 2018; Rios & Itti, 2018; Zhang et al., 2019).”
Understanding the Role of Training Regimes in Continual Learning (2020, background): “The first group consists of replay based methods that build and store a memory of the knowledge learned from old tasks [48, 62, 82, 68, 63], known as experience replay.”
Orthogonal Gradient Descent for Continual Learning (2019, background): “Shin et al. (2017); Kamra et al. (2017); Zhang et al. (2019); Rios and Itti (2018) learned a generative model to capture the data distribution of previous tasks, along with the current task’s data to train the new model so that the forgetting can be alleviated.”
Continual Learning with Filter Atom Swapping (2022, methodology): “(De Lange et al., 2019) assumes it is feasible to access data from previous tasks by having a ﬁxed-size memory or a generative model able to produces samples from previous tasks (Lopez-Paz & Ranzato, 2017; Riemer et al., 2018; Rios & Itti, 2018; Shin et al., 2017).”
TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation (2023, background): “Some store the knowledge of the previous tasks, known as experience replay [35, 45, 49, 46].”
Task-agnostic Continual Learning with Hybrid Probabilistic Models (2021, unknown): “Replay - based methods store and rephrase a memory of the examples or knowledge learned so far [Rebuﬃ et al., 2016, Lopez-Paz and Ranzato, 2017, Shin et al., 2017, Riemer et al., 2018, Rios and Itti, 2018, Zhang et al., 2019, Chaudhry et al., 2020, Balaji et al., 2020].”