On the Soft-Subnetwork for Few-shot Class Incremental Learning

By Haeyong Kang, Jaehong Yoon, Sultan Rizky Hikmawan Madjid, Sung Ju Hwang, Chang-Dong Yoo. 2022

65 citations recorded by the source corpus.

Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive performance of the dense network, we propose a few-shot class incremental learning (FSCIL) method referred to as \emph{Soft-SubNetworks (SoftNet)}. Our objective is to learn a sequence of sessions incrementally, where each session only includes a few training instances per class while preserving the knowledge of the previously learned ones. SoftNet jointly learns the model weights and adaptive non-binary soft masks at a base training session in which each mask consists of the major and minor subnetwork; the former aims to minimize catastrophic forgetting during training, and the latter aims to avoid overfitting to a few samples in each new training session. We provide comprehensive empirical validations demonstrating that our SoftNet effectively tackles the few-shot incremental learning problem by surpassing the performance of state-of-the-art baselines over benchmark datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xin Ning
2. Weijun Li
3. Prayag Tiwari
4. Hang Ran
5. Lusi Li
6. Song Tian

In Computer Vision and Pattern Recognition

1. Xin Ning
2. Weijun Li
3. Prayag Tiwari
4. Hang Ran
5. Lusi Li
6. Song Tian

Citations in the corpus

• A Survey on Few-Shot Class-Incremental Learning (2023, background): “In the latest study, Yoon et al. (2023) explores a masking-based method in network structure.”