Exemplar-Supported Generative Reproduction for Class Incremental Learning

By Chen He, Ruiping Wang, Shiguang Shan, Xilin Chen. 2018

90 citations recorded by the source corpus.

Subfields: Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

1. Yilun Du
2. Shuang Li
3. Igor Mordatch
4. Zhangyang Wang
5. Jun Zhu
6. Antonio Torralba
7. Shiyu Chang
8. Qi Tian

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Liyuan Wang
5. Xingxing Zhang
6. Jun Zhu
7. Hang Su
8. Qiwen Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Weight regularization [25], [55], [161], [162] and experience replay [25], [163] have been shown to be effective in mitigating catastrophic forgetting of generative models.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “In image classification, He et al. [208] proposed an Ex-emplar Supported Generative Reproduction (ESGR) method, which leverages both generated data instances and stored real data instances to better mitigate catastrophic forgetting.”
• RECALL: Replay-based Continual Learning in Semantic Segmentation (2021, methodology): “We remark that exact correspondence between GAN’s conditioning class space and semantic segmentation category set is not required by our replay strategy.”
• Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…adversarial network (GAN) (Goodfellow et al., 2014) to generate samples from previously learned data (Draelos et al., 2017; Kemker & Kanan, 2018; Robins, 1995, 1996; Ostapenko et al., 2019; Shin, Lee, Kim, & Kim, 2017; He et al., 2018; French, 1997; Atkinson, McCane, Szymanski, & Robins, 2018).”
• Few-Shot Lifelong Learning (2021, background): “This setting is also known as the class-incremental setting of lifelong learning (He et al. 2018; Rebuffi et al. 2017; Castro et al. 2018).”
• GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning (2021, methodology): “We can view the inducing inputs, which are learned per class, as ’exemplars’ of the base classes, a commonly used practice in incremental learning studies (He et al., 2018).”
• Lifelong Graph Learning (2020, neutral): “Rehearsal approaches with synthetic data based on generative adversary networks (GAN) were used to reduce the dependence on old data [14, 40, 50, 53].”
• MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, background): “Rehearsal strategies [15], [16], [17], [54], [55], [56], [57], [58], [59], [60], [61], [62] replay old tasks information when learning the new task, and the past knowledge is memorized by storing old tasks’ exemplars or old tasks data distribution via generative models.”
• Few-Shot Class-Incremental Learning (2020, methodology): “In this paper, we study the FSCIL problem under the multi-class scenario, where we treat FSCIL as a particular case of the class-incremental learning (CIL) [32, 2, 10, 13, 48].”
• Memory-Efficient Incremental Learning Through Feature Adaptation (2020, methodology): “ted images. These methods use generative models (GANs [10]) to generate fake images that mimic the past data, and use the generated images when learning the network for new classes [36,40]. He et al. [13] use multiple generators to increase capacity as new sets of classes become available. A major d”
• ScaIL: Classifier Weights Scaling for Class Incremental Learning (2020, methodology): “Generative Adversarial Networks were also considered as a mean to generate image exemplars for past classes instead of storing them directly [7].”
• IL2M: Class Incremental Learning With Dual Memory (2019, methodology): “As a result, the usual evaluation protocols include two [5, 8, 12, 24] or three [13] datasets which are generally smaller than the ones used here.”
• Class-incremental Learning via Deep Model Consolidation (2019, background): “Existing endto-end trainable pseudo-rehearsal methods do not produce competitive results [49] unless supported by real exemplars [40, 15].”
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020, background): “In spite of this observation, knowledge distillation is commonly used to tackle catastrophic forgetting [10, 16, 20, 23, 35, 41, 65].”
• Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning (2021, background): “(Castro et al., 2018; He et al., 2018; Javed & Shafait, 2018; Rebuffi et al., 2017) are early attempts for undertaking the forgetting dilemma by introducing knowledge distillation regularization (Hinton et al.”
• Selective Replay Enhances Learning in Online Continual Analogical Reasoning (2021, unknown): “Standard replay approaches store explicit images or features in a memory buffer, while generative replay (or pseudo-rehearsal) methods train a generative model to generate previous samples [25, 36, 46, 60, 75].”
• Class-Incremental Learning: A Survey (2023, background): “There are several kinds of data replay to help the model recover former knowledge, i.e. , image [82], [128], feature [38], [50] and generative replay [41], [42].”
• Network Adaptation Strategies for Learning New Classes without Forgetting the Original Ones (2019, background): “[10] described a combined approach that both stores samples from the original classes and trains a generative model based on the original classes.”