Part-aware Prototype Network for Few-shot Semantic Segmentation

By Yongfei Liu, Xiangyi Zhang, Songyang Zhang, Xuming He. 2020

406 citations recorded by the source corpus.

Few-shot semantic segmentation aims to learn to segment new object classes with only a few annotated examples, which has a wide range of real-world applications. Most existing methods either focus on the restrictive setting of one-way few-shot segmentation or suffer from incomplete coverage of object regions. In this paper, we propose a novel few-shot semantic segmentation framework based on the prototype representation. Our key idea is to decompose the holistic class representation into a set of part-aware prototypes, capable of capturing diverse and fine-grained object features. In addition, we propose to leverage unlabeled data to enrich our part-aware prototypes, resulting in better modeling of intra-class variations of semantic objects. We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes based on labeled and unlabeled images. Extensive experimental evaluations on two benchmarks show that our method outperforms the prior art with a sizable margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, methodology): “For example, few-shot learning [21, 43, 50, 55, 70, 77, 84] and active learning [8, 62, 66, 67, 83] are proposed to perform segmentation with as few pixel-wise labels as possible.”
Progressive One-shot Human Parsing (2020, unknown): “Recently, the metric was further improved by decomposing the class representations by part-aware prototypes in (Liu et al. 2020b).”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, methodology): “We evaluate the proposed model on PASCAL-5, COCO20, and FSS-1000 and compare the results with recent methods [4, 37, 46, 54, 61, 63, 70, 74, 75, 86].”
Prior Guided Feature Enrichment Network for Few-Shot Segmentation (2020, unknown): “Feature processing can be accomplished by generating weights for the classifier [33], [41], cosinesimilarity calculation [5], [45], [23], or convolutions [15], [54], [49], [9], [1] to generate the final prediction.”
Generalized Few-shot Semantic Segmentation (2020, background): “Few-Shot Segmentation (FSSeg) places semantic segmentation in the few-shot scenario [2,8,11,15,18,20,21,24,27,28,33,34,38,41,48,51,52,57, 61, 62, 66, 67], where dense pixel labeling is performed on new classes with only a few support samples.”
Visual Prompting via Image Inpainting (2022, methodology): “In this setting, the algorithm is trained on a labeled dataset of base classes, from which it should transfer to a set of novel classes given only a few training examples (like 10 or 30) [36, 27, 32, 53, 55, 48, 59, 2].”
Novel Class Discovery in Semantic Segmentation (2021, methodology): “Following few-shot semantic segmentation [18, 23], we split the base and novel classes as PASCAL-5 dataset [27].”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, unknown): “8 PPNet [261] ECCV’20 ResNet-50 Metric 51.”
Image Segmentation Using Text and Image Prompts (2021, background): “Several approaches focus on the modeling of prototypes [37, 38, 39].”
Few-Shot Segmentation via Cycle-Consistent Transformer (2021, methodology): “Recent works attempt to generate multiple prototypes via EM algorithm or K-means clustering [36, 16], in order to extract more abundant information from support images.”
Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, methodology): “6 one can see that ASR outperforms PPNet [24] with a Method Pascal-5 Pascal-5(1) Pascal-5(2) Pascal-5(3) Mean PPNet [24] 47.”
Learning What Not to Segment: A New Perspective on Few-Shot Segmentation (2022, methodology): “Following the previous works [34, 51, 61], we adopt mean intersection-over-union (mIoU) and foreground-background IoU (FB-IoU) as the evaluation metrics for experiments.”