Incremental Learning Using Conditional Adversarial Networks

By Ye Xiang, Ying Fu, Pan Ji, Hua Huang. 2019

193 citations recorded by the source corpus.

Incremental learning using Deep Neural Networks (DNNs) suffers from catastrophic forgetting. Existing methods mitigate it by either storing old image examples or only updating a few fully connected layers of DNNs, which, however, requires large memory footprints or hurts the plasticity of models. In this paper, we propose a new incremental learning strategy based on conditional adversarial networks. Our new strategy allows us to use memory-efficient statistical information to store old knowledge, and fine-tune both convolutional layers and fully connected layers to consolidate new knowledge. Specifically, we propose a model consisting of three parts, i.e., a base sub-net, a generator, and a discriminator. The base sub-net works as a feature extractor which can be pre-trained on large scale datasets and shared across multiple image recognition tasks. The generator conditioned on labeled embeddings aims to construct pseudo-examples with the same distribution as the old data. The discriminator combines real-examples from new data and pseudo-examples generated from the old data distribution to learn representation for both old and new classes. Through adversarial training of the discriminator and generator, we accomplish the multiple continuous incremental learning. Comparison with the state-of-the-arts on public CIFAR-100 and CUB-200 datasets shows that our method achieves the best accuracies on both old and new classes while requiring relatively less memory storage.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “For example, FearNet [210] and ILCAN [461] additionally preserves statistical information of the old features acquired from a pre-trained feature extractor, while GAN-Memory [80] continually adjusts a pre-trained generative model with taskspecific parameters.”
ALIFE: Adaptive Logit Regularizer and Feature Replay for Incremental Semantic Segmentation (2022, methodology): “To avoid these problems, recent works employ a pseudo replay scheme that synthesizes images [20, 31, 34] or latent features [38] of previous categories.”
Continual Learning through Retrieval and Imagination (2022, methodology): “Generative-based methods synthesize previous data produced by generative models and replay the synthesized data (Xiang et al. 2019; van de Ven et al. 2020).”
Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “Other works consider saving embeddings instead of raw images [38], using generative models for data rehearsal [39], task-wise adaptation [40], and output normalization [18], [41], [42].”
Bridging Non Co-occurrence with Unlabeled In-the-wild Data for Incremental Object Detection (2021, neutral): “They usually keep a small number of exemplars [22, 29, 3], or generate synthetic images [26, 19] or features [12, 30] to achieve this purpose.”
Online Continual Learning Via Candidates Voting (2021, methodology): “Replay-based methods are shown to be effective for maintaining learned knowledge by either using the original data as exemplars [35, 34, 29, 6, 27, 33, 2, 3, 8, 37, 9, 7] or synthetic data and statistics [38, 40, 43, 22].”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “[191] propose storing statistical information of previous data instances, such as mean and covariance, which are fed to a conditional GAN to generate pseudo data instances.”
Co-Transport for Class-Incremental Learning (2021, background): “Other works selectively save exemplars from old classes and rehearsal them when learning new tasks [16, 35, 48].”
EEC: Learning to Encode and Regenerate Images for Continual Learning (2021, methodology): “To avoid storing old class images, some approaches store features from the last fully-connected layer of the neural networks (Xiang et al., 2019; Hayes & Kanan, 2020; Ayub & Wagner, 2020b;d).”
ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning (2021, background): “In the next task or batch, the generator replays conditional samples with the new training samples to update the generator and the classifier [35, 43, 45, 18].”
Lifelong Graph Learning (2020, neutral): “Rehearsal approaches with synthetic data based on generative adversary networks (GAN) were used to reduce the dependence on old data [14, 39, 49, 52].”
MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, background): “Incremental learning is usually conducted under the task-incremental [72], [73] or the class-incre-mental learning scenarios [2], [70], [74], [75], [76], [77].”
Generative Feature Replay For Class-Incremental Learning (2020, methodology): “In contrast, FearNet [17] uses a generative autoencoder for memory replay and [53] generates intermediate features.”
Memory-Efficient Incremental Learning Through Feature Adaptation (2020, methodology): “We preserve and adapt feature descriptors to new feature spaces as the network is trained incrementally. This allows training the network from scratch and does not depend on a pre-trained model as in [26,55]. Compared to existing rehearsal methods, our method has a signicantly lower memory footprint”
SS-IL: Separated Softmax for Incremental Learning (2020, methodology): “For general continual learning, there has been three main approaches; 1) regularizationbased [22, 34, 9, 2, 19], 2) dynamic architecture-based [29, 33, 24, 17], and 3) exemplar/replay-memory based [26, 10, 11, 30, 21, 32, 30, 21, 32] methods.”
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification (2020, methodology): “images [30], [53] or features [37], [54] (pseudo-rehearsal).”