Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation

By Yixiong Zou, Shanghang Zhang, Yuhua Li, Rui Li. 2022

98 citations recorded by the source corpus.

Few-shot class-incremental learning (FSCIL) is designed to incrementally recognize novel classes with only few training samples after the (pre-)training on base classes with sufficient samples, which focuses on both base-class performance and novel-class generalization. A well known modification to the base-class training is to apply a margin to the base-class classification. However, a dilemma exists that we can hardly achieve both good base-class performance and novel-class generalization simultaneously by applying the margin during the base-class training, which is still under explored. In this paper, we study the cause of such dilemma for FSCIL. We first interpret this dilemma as a class-level overfitting (CO) problem from the aspect of pattern learning, and then find its cause lies in the easily-satisfied constraint of learning margin-based patterns. Based on the analysis, we propose a novel margin-based FSCIL method to mitigate the CO problem by providing the pattern learning process with extra constraint from the margin-based patterns themselves. Extensive experiments on CIFAR100, Caltech-USCD Birds-200-2011 (CUB200), and miniImageNet demonstrate that the proposed method effectively mitigates the CO problem and achieves state-of-the-art performance.

Subfields: Artificial Intelligence, Cancer Research, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xin Ning
2. Weijun Li
3. Prayag Tiwari
4. Hang Ran
5. Yuhua Li
6. Yixiong Zou
7. Ruixuan Li
8. Haichen Zhou

In Representation & Self-Supervised Learning

1. Xin Ning
2. Weijun Li
3. Prayag Tiwari
4. Hang Ran
5. Lusi Li
6. Song Tian

Citations in the corpus

• A Survey on Few-Shot Class-Incremental Learning (2023, methodology): “Zou et al. (2022) divided FSCIL into metric-based and fine-tuning-based methods.”
• Compositional Few-Shot Class-Incremental Learning (2024, methodology): “To compare with current works, we implement CLOM (Zou et al., 2022) as the state-of-the-art method that has the highest last-session accuracy in Tab.5.”