Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning

By Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip H. S. Torr, Dacheng Tao. 2023

177 citations recorded by the source corpus.

Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. Finetuning the backbone or adjusting the classifier prototypes trained in the prior sessions would inevitably cause a misalignment between the feature and classifier of old classes, which explains the well-known catastrophic forgetting problem. In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF). It corresponds to an optimal geometric structure for classification due to the maximized Fisher Discriminant Ratio. We propose a neural collapse inspired framework for FSCIL. A group of classifier prototypes are pre-assigned as a simplex ETF for the whole label space, including the base session and all the incremental sessions. During training, the classifier prototypes are not learnable, and we adopt a novel loss function that drives the features into their corresponding prototypes. Theoretical analysis shows that our method holds the neural collapse optimality and does not break the feature-classifier alignment in an incremental fashion. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances. Code address: https://github.com/NeuralCollapseApplications/FSCIL

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Keon-Hee Park
2. Kyungwoo Song
3. Xin Ning
4. Weijun Li
5. Prayag Tiwari
6. Hang Ran
7. Yuhua Li
8. Yixiong Zou

In Cancer Research

1. Gyeong-Moon Park

In Computer Vision and Pattern Recognition

1. Keon-Hee Park
2. Kyungwoo Song
3. Xin Ning
4. Weijun Li
5. Prayag Tiwari
6. Hang Ran
7. Yuhua Li
8. Yixiong Zou

Citations in the corpus

• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “In image semantic segmentation, Wu et al. [211] proposed to store feature statistics of previous images to help train the image generator.”
• A Survey on Few-Shot Class-Incremental Learning (2023, methodology): “With the exception of some early methods (TOPIC (Tao et al., 2020), SPPR (Zhu et al., 2021b), VAE-based (Cheraghian et al., 2021b)), most methods have an accuracy of 70% to 80% on the base dataset, and few methods have an accuracy above 80% on the base dataset (F2M (SHI et al., 2021), DSN (Yang et a”
• Pre-trained Vision and Language Transformers are Few-Shot Incremental Learners (2024, methodology): “To address these challenges, previous studies mainly have utilized shallow models like Resnet-18 [12, 39, 40].”
• Compositional Few-Shot Class-Incremental Learning (2024, methodology): “For mini ImageNet, we follow NC-FSCIL (Yibo Yang, 2023) to utilize ResNet12 (He et al., 2016) as the backbone network, and we set λ 1 = λ 2 = 2 .”