SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation

By Xiaolin Zhang, Yunchao Wei, Yi Yang, Thomas S. Huang. IEEE Transactions on Cybernetics, 2018

584 citations recorded by the source corpus.

One-shot image semantic segmentation poses a challenging task of recognizing the object regions from unseen categories with only one annotated example as supervision. In this article, we propose a simple yet effective similarity guidance network to tackle the one-shot (SG-One) segmentation problem. We aim at predicting the segmentation mask of a query image with the reference to one densely labeled support image of the same category. To obtain the robust representative feature of the support image, we first adopt a masked average pooling strategy for producing the guidance features by only taking the pixels belonging to the support image into account. We then leverage the cosine similarity to build the relationship between the guidance features and features of pixels from the query image. In this way, the possibilities embedded in the produced similarity maps can be adopted to guide the process of segmenting objects. Furthermore, our SG-One is a unified framework that can efficiently process both support and query images within one network and be learned in an end-to-end manner. We conduct extensive experiments on Pascal VOC 2012. In particular, our SG-One achieves the mIoU score of 46.3%, surpassing the baseline methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1ecd20f7fc34344e396825d27bc5a9871ab0d0c2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

AMP: Adaptive Masked Proxies for Few-Shot Segmentation (2019, methodology): “[41] use the output of their pooling layer to compute a guidance to the base network.”
Prototype-based Incremental Few-Shot Segmentation (2020, background): “Similarly to iFSS, Few-Shot Segmentation (FSS) [8, 36, 42, 43, 49, 56, 58] aims to segment new classes, given few images depicting them.”
Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation (2022, unknown): “In this branch, masked average pooling (MAP) [42] is used to generate prototype (centroid) pr ∈ R N × C of each proposal, where C is the number of channels of feature map.”
PANet: Few-Shot Image Semantic Segmentation With Prototype Alignment (2019, methodology): “Late fusion directly masks over the feature maps to produce foreground/background features separately [28, 16].”
Repurposing GANs for One-Shot Semantic Part Segmentation (2021, background): “Similarity guidance network [55] masks off the background in the support image, then finds the pixels in the query branch with similar foreground features.”
Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need? (2020, background): “More recently, the dual-branch setting has been unified into a single-branch, employing the same embedding function for both the support and query sets [42, 30, 37, 38, 21].”
Generalized Few-shot Semantic Segmentation (2020, background): “Few-Shot Segmentation (FSSeg) places semantic segmentation in the few-shot scenario [2,8,11,15,18,20,21,24,27,28,33,34,38,41,48,51,52,57, 61, 62, 66, 67], where dense pixel labeling is performed on new classes with only a few support samples.”
Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (2021, methodology): “networks [40, 42, 32, 29, 4] merely generate a single prototype by masked average pooling as shown in Figure 1(a), thus losing information as well as discriminability.”
CRNet: Cross-Reference Networks for Few-Shot Segmentation (2020, methodology): “To achieve a categoryrelevant embedding, previous works opt to filter out the background regions in the input images [14, 17] or in the feature representations[26, 27].”
Few-Shot Segmentation via Cycle-Consistent Transformer (2021, methodology): “SG-One [39] leverage the cosine similarity map between the single support prototype and query features to guide the prediction.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, unknown): “2 SG-One [267] IEEE’20 VGG-16 Metric ✓ ✓ ✓ 14.”
Feature Weighting and Boosting for Few-Shot Segmentation (2019, methodology): “Our approach extends SG-One with the two contributions specified in the next section.”
Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, methodology): “The category labels for the Method 1-shot 5-shot SG-One [47] 63.”
Prototype Mixture Models for Few-shot Semantic Segmentation (2020, methodology): “is then used to tune the segmentation process of an image in the query branch. In PL [17], the idea of prototypical networks was employed to tackle few-shot segmentation using metric learning. SG-One [16] also used a prototype vector to guide semantic segmentation procedure. To obtain the squeezed r”
Few-Shot Semantic Segmentation with Democratic Attention Networks (2020, methodology): “Existing deep models for few-shot semantic segmentation are mainly built upon a two-branch structure [28, 25, 41, 40, 11] which includes a support branch and”
Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation (2019, methodology): “…metric of mean class-wise IoU adopted in [18] Model (5-shot) IoU co-FCN [15] 60.2 OSLSM [18] 61.5 A-MCG-Conv-LSTM [9] 62.2 PL [3] 62.3 SG-One [27] 65.9 Reduced-DFCN8s [19] 66.0 CANet [26] 69.6 PGNet-Graph-MS 70.5 (b) Evaluation metric of IoU adopted in [15] Table 8: Comparison with the…”
Part-aware Prototype Network for Few-shot Semantic Segmentation (2020, methodology): “Then we present our experimental analysis on PASCAL-5i dataset in Sec.”
Self-Supervised Tuning for Few-Shot Segmentation (2020, methodology): “, 2017], SG-One [Zhang et al., 2018], PAnet [Wang et al.”