Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation

By Guangchen Shi, Yirui Wu, J. Liu, Shaohua Wan, Wenhai Wang, Tong Lu. 2022

46 citations recorded by the source corpus.

Incremental few-shot semantic segmentation (IFSS) targets at incrementally expanding model's capacity to segment new class of images supervised by only a few samples. However, features learned on old classes could significantly drift, causing catastrophic forgetting. Moreover, few samples for pixel-level segmentation on new classes lead to notorious overfitting issues in each learning session. In this paper, we explicitly represent class-based knowledge for semantic segmentation as a category embedding and a hyper-class embedding, where the former describes exclusive semantical properties, and the latter expresses hyper-class knowledge as class-shared semantic properties. Aiming to solve IFSS problems, we present EHNet, i.e., Embedding adaptive-update and Hyper-class representation Network from two aspects. First, we propose an embedding adaptive-update strategy to avoid feature drift, which maintains old knowledge by hyper-class representation, and adaptively update category embeddings with a class-attention scheme to involve new classes learned in individual sessions. Second, to resist overfitting issues caused by few training samples, a hyper-class embedding is learned by clustering all category embeddings for initialization and aligned with category embedding of the new class for enhancement, where learned knowledge assists to learn new knowledge, thus alleviating performance dependence on training data scale. Significantly, these two designs provide representation capability for classes with sufficient semantics and limited biases, enabling to perform segmentation tasks requiring high semantic dependence. Experiments on PASCAL-5i and COCO datasets show that EHNet achieves new state-of-the-art performance with remarkable advantages.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Hang Su
5. Qiwen Wang
6. Jun Zhu
7. Ziwei Liu
8. Liyuan Wang

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Jun Zhu
4. Da-Wei Zhou
5. Hang Su
6. Liyuan Wang
7. Ziwei Liu
8. Qiwen Wang

In Radiology, Nuclear Medicine and Imaging

1. Da-Wei Zhou
2. De-chuan Zhan
3. Han-Jia Ye
4. Liyuan Wang
5. Jun Zhu
6. Hang Su
7. Ziwei Liu
8. Xingxing Zhang

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Qiwen Wang
8. Jun Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “In addition to the regular CSS, other relevant settings include unsupervised domain adaptation for CSS [394], where a pre-trained model adapts to the unannotated data of the target domain without using the annotated data of the source domain; incremental few-shot semantic segmentation [379], which p”
• A Survey on Few-Shot Class-Incremental Learning (2023, background): “Shi et al. (2022) proposed the Em-bedding adaptive-update and Hyper-class representation Network (EHNet) for incremental few-shot learning.”
• Class-Incremental Learning: A Survey (2023, background): “Recent works in CSS also involve incremental learning in more challenging scenarios, e.g. , learning with few-shot annotations [283], [284], different domains [285], [286], continual instance segmentation [287], [288] and weakly semantic segmentation [289].”