Prototype-based Incremental Few-Shot Segmentation

By Fabio Cermelli, Massimiliano Mancini, Yongqin Xian, Zeynep Akata, Barbara Caputo. 2020

44 citations recorded by the source corpus.

Semantic segmentation models have two fundamental weaknesses: i) they require large training sets with costly pixel-level annotations, and ii) they have a static output space, constrained to the classes of the training set. Toward addressing both problems, we introduce a new task, Incremental Few-Shot Segmentation (iFSS). The goal of iFSS is to extend a pretrained segmentation model with new classes from few annotated images and without access to old training data. To overcome the limitations of existing models iniFSS, we propose Prototype-based Incremental Few-Shot Segmentation (PIFS) that couples prototype learning and knowledge distillation. PIFS exploits prototypes to initialize the classifiers of new classes, fine-tuning the network to refine its features representation. We design a prototype-based distillation loss on the scores of both old and new class prototypes to avoid overfitting and forgetting, and batch-renormalization to cope with non-i.i.d.few-shot data. We create an extensive benchmark for iFSS showing that PIFS outperforms several few-shot and incremental learning methods in all scenarios.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Qiwen Wang
5. Ziwei Liu
6. Zhiyuan Qi
7. Prayag Tiwari
8. Xin Ning

Citations in the corpus

• Incremental Learning in Semantic Segmentation from Image Labels (2021, neutral): “Finally, [9] proposed the incremental few-shot segmentation setting, where only a few images to learn new classes are provided.”
• Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation (2022, methodology): “Recent CSS methods such as SDR [24] and PIFS [3] use prototypes to regularize features for reducing the forgetting of old classes.”
• CoMFormer: Continual Learning in Semantic and Panoptic Segmentation (2022, unknown): “Traditionally, it has been studied in the context of image classification [6, 17, 19, 29, 33, 43, 45, 55] but there is a growing interest in its applications in segmentation [2–5,15,16,38–40,59].”
• A Survey on Few-Shot Class-Incremental Learning (2023, background): “Cermelli et al. (2021) proposed the first attempt to solve incremental few-shot semantic segmentation.”
• Class-Incremental Learning: A Survey (2023, background): “Recent works in CSS also involve incremental learning in more challenging scenarios, e.g. , learning with few-shot annotations [283], [284], different domains [285], [286], continual instance segmentation [287], [288] and weakly semantic segmentation [289].”