Self-Supervised Tuning for Few-Shot Segmentation

By Kai Zhu, Wei Zhai, Zhengjun Zha, Yang Cao. 2020

30 citations recorded by the source corpus.

Few-shot segmentation aims at assigning a category label to each image pixel with few annotated samples. It is a challenging task since the dense prediction can only be achieved under the guidance of latent features defined by sparse annotations. Existing meta-learning based method tends to fail in generating category-specifically discriminative descriptor when the visual features extracted from support images are marginalized in embedding space. To address this issue, this paper presents an adaptive tuning framework, in which the distribution of latent features across different episodes is dynamically adjusted based on a self-segmentation scheme, augmenting category-specific descriptors for label prediction. Specifically, a novel self-supervised inner-loop is firstly devised as the base learner to extract the underlying semantic features from the support image. Then, gradient maps are calculated by back-propagating self-supervised loss through the obtained features, and leveraged as guidance for augmenting the corresponding elements in the embedding space. Finally, with the ability to continuously learn from different episodes, an optimization-based meta-learner is adopted as outer loop of our proposed framework to gradually refine the segmentation results. Extensive experiments on benchmark PASCAL-5i and COCO-20i datasets demonstrate the superiority of our proposed method over state-of-the-art.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Li Jiang
2. Hengshuang Zhao
3. Yang Cao
4. Wei Zhai
5. Kai Zhu
6. Zhengjun Zha
7. Zhuotao Tian
8. Binghao Liu

In Computer Vision and Pattern Recognition

1. Qixiang Ye
2. Luc Van Gool
3. Jiaya Jia
4. Zhengjun Zha
5. Hengshuang Zhao
6. Jitendra Malik
7. Jianbin Jiao
8. Yi Yang

Citations in the corpus

• Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning (2021, methodology): “Meta-based methods [7, 22, 37, 10] aims to learn a learning strategy to adjust well to new samples.”
• Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, methodology): “SST [49] and SimProp [9] respectively introduced self-supervised finetuning and similarity propagation, which leverage the category-specific semantic constraints to reduce semantic aliasing.”
• Generalized Few-shot Semantic Segmentation (2020, background): “Few-Shot Segmentation (FSSeg) places semantic segmentation in the few-shot scenario [2,8,11,15,18,20,21,24,27,28,33,34,38,41,48,51,52,57, 61, 62, 66, 67], where dense pixel labeling is performed on new classes with only a few support samples.”