MetaFSCIL: A Meta-Learning Approach for Few-Shot Class Incremental Learning

By Zhixiang Chi, Li Gu, Huan Liu, Yang Wang, Yuanhao Yu, Jingshan Tang. 2022

188 citations recorded by the source corpus.

In this paper, we tackle the problem of few-shot class incremental learning (FSCIL). FSCIL aims to incrementally learn new classes with only a few samples in each class. Most existing methods only consider the incremental steps at test time. The learning objective of these methods is often hand-engineered and is not directly tied to the objective (i.e. incrementally learning new classes) during testing. Those methods are sub-optimal due to the misalignment between the training objectives and what the methods are expected to do during evaluation. In this work, we proposed a bi-level optimization based on meta-learning to directly optimize the network to learn how to incrementally learn in the setting of FSCIL. Concretely, we propose to sample sequences of incremental tasks from base classes for training to simulate the evaluation protocol. For each task, the model is learned using a meta-objective such that it is capable to perform fast adaptation without forgetting. Furthermore, we propose a bi-directional guided modulation, which is learned to automatically modulate the activations to reduce catastrophic forgetting. Extensive experimental results demonstrate that the proposed method outperforms the baseline and achieves the state-of-the-art results on CIFARIOO, MiniImageNet, and CUB200 datasets.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Do-Yeon Kim
3. Yang Wang
4. Jaekyun Moon
5. Jun Seo
6. Zhouchen Lin
7. Prayag Tiwari
8. Philip H. S. Torr

In Cancer Research

1. Dacheng Tao
2. Philip H. S. Torr

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Philip H. S. Torr
4. Xiaogang Wang
5. Ming-Hsuan Yang
6. Cordelia Schmid
7. Jitendra Malik
8. Andrew Zisserman

In Meta & Continual Learning

1. Dacheng Tao
2. Philip H. S. Torr
3. Jaekyun Moon
4. Dong-Jun Han
5. Do-Yeon Kim
6. Jun Seo
7. Jiangpeng He
8. Feng Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “…graph for knowledge distillation [250], selectively updating only the unimportant parameters [251] or stabilizing the important parameters [252], updating parameters within the flat region of loss landscape [84], meta-learning of a good initialization [253], as well as generative replay [254].”
• Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay (2022, methodology): “[9] proposes a bi-level optimization technique to learn how to incrementally learn in the setting of FSCIL.”
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning (2023, background): “A popular choice is to use meta learning (Yoon et al., 2020; Chi et al., 2022; Zhou et al., 2022b).”
• A Survey on Few-Shot Class-Incremental Learning (2023, methodology): “In the majority of FSCIL experiments (Cheraghian et al., 2021c; Chi et al., 2022; Liu et al., 2022b; Peng et al., 2022; SHI et al., 2021; Tao et al., 2020; Yang et al., 2022; Zhang et al., 2021; Zheng and Zhang, 2021; Zhou et al., 2022b; Zhu et al., 2021a), the three image classification datasets…”
• Warping the Space: Weight Space Rotation for Class-Incremental Few-Shot Learning (2023, methodology): “Additionally, we further compare our method with more recent works (Zhou et al., 2022b; Chi et al., 2022; Hersche et al., 2022; Liu et al., 2022) on mini ImageNet dataset.”