Few-Shot Class-Incremental Learning

By Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, Yihong Gong. 2020

556 citations recorded by the source corpus.

The ability to incrementally learn new classes is crucial to the development of real-world artificial intelligence systems. In this paper, we focus on a challenging but practical few-shot class-incremental learning (FSCIL) problem. FSCIL requires CNN models to incrementally learn new classes from very few labelled samples, without forgetting the previously learned ones. To address this problem, we represent the knowledge using a neural gas (NG) network, which can learn and preserve the topology of the feature manifold formed by different classes. On this basis, we propose the TOpology-Preserving knowledge InCrementer (TOPIC) framework. TOPIC mitigates the forgetting of the old classes by stabilizing NG's topology and improves the representation learning for few-shot new classes by growing and adapting NG to new training samples. Comprehensive experimental results demonstrate that our proposed method significantly outperforms other state-of-the-art class-incremental learning methods on CIFAR100, miniImageNet, and CUB200 datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “According to the provided X t and Y t in each D t , continual learning is further extended to zero-shot [23], few-shot [24], semi-supervised [25], open-world [26] and un-/self-supervised [27], [28] scenarios.”
S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learning (2023, methodology): “For this, to mitigate catastrophic forgetting of the base classes, their class prototypes are stored as is the common practice [35][44].”
First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning (2023, methodology): “To quantify the forgetting behavior, we use a scaled modification of the performance dropping rate [29], namely percent performance dropping rate (PPDR), defined as PPDR = 100 × A 1 −A S A 1 % , where A 1 denotes test accuracy after the first session, and A S the test accuracy after the last…”
Towards Label-Efficient Incremental Learning: A Survey (2023, methodology): “1-shot) [ Tao et al. , 2020].”
Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation (2022, methodology): “As split in [31], 60 classes are chosen as the base-session classes, and 40 classes are used as novel classes.”
Few-Shot Continual Active Learning by a Robot (2022, background): “As the FSCIL approaches are not designed for FoCAL, for a fair compairson we tested GBCL for the standard FSCIL setup [9] on the CIFAR-100 dataset [43].”
Few-Shot Class-Incremental Learning from an Open-Set Perspective (2022, neutral): “We compare our method with the SOTA methods [17,1,11,19,4,5,23] on three datasets.”
Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay (2022, methodology): “Figure 1 presents the comparison between TOPIC and replay-based approaches, from which we can observe that all the replay-based approaches outperform TOPIC [27] by a significant margin.”
Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, methodology): “An intuitive way for few-shot class-incremental learning is to directly adopt CIL methods, which get poor performance [21], [22].”
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches (2022, methodology): “Amongst its many variants, the recent Few-Shot Class Incremental Learning (FSCIL) [54] is the most realistic where it also dictates the model to learn new classes with very few examples, the same as humans do.”
Forward Compatible Few-Shot Class-Incremental Learning (2022, background): “We report the results of compared methods from [9] and [12].”
Memory Replay with Data Compression for Continual Learning (2022, positive): “…performance in class-incremental learning (Liu et al., 2021a; Hu et al., 2021) and in numerous other continual learning scenarios, such as audio tasks (Ehret et al., 2020), few-shot (Tao et al., 2020b), semi-supervised (Wang et al., 2021a), and unsupervised continual learning (Khare et al., 2021).”
Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima (2021, methodology): “The study of incremental few-shot learning has just started [47, 41, 60, 9, 8, 34, 59].”
Subspace Regularizers for Few-Shot Class Incremental Learning (2021, methodology): “As noted by past work, a more realistic evaluation of models’ ability to rapidly acquire new concepts should consider their ability to discriminate among both new concepts and old ones, a problem usually referred to as few-shot class-incremental learning (FSCIL) 2 (Tao et al., 2020b).”
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, background): “The usual way to evaluate class incremental learning [7, 15, 1] is to continuously measure performance of the model in the growing joint space”