Feature Weighting and Boosting for Few-Shot Segmentation

By Khoi Duc Minh Nguyen, Sinisa Todorovic. 2019

410 citations recorded by the source corpus.

This paper is about few-shot segmentation of foreground objects in images. We train a CNN on small subsets of training images, each mimicking the few-shot setting. In each subset, one image serves as the query and the other(s) as support image(s) with ground-truth segmentation. The CNN first extracts feature maps from the query and support images. Then, a class feature vector is computed as an average of the support's feature maps over the known foreground. Finally, the target object is segmented in the query image by using a cosine similarity between the class feature vector and the query's feature map. We make two contributions by: (1) Improving discriminativeness of features so their activations are high on the foreground and low elsewhere; and (2) Boosting inference with an ensemble of experts guided with the gradient of loss incurred when segmenting the support images in testing. Our evaluations on the PASCAL-5i and COCO-20i datasets demonstrate that we significantly outperform existing approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Language-driven Semantic Segmentation (2022, methodology): “Few-shot approaches (Shaban et al., 2017; Rakelly et al., 2018; Siam et al., 2019; Wang et al., 2019; Zhang et al., 2019; Nguyen & Todorovic, 2019; Liu et al., 2020b; Wang et al., 2020; Tian et al., 2020; Boudiaf et al., 2021; Min et al., 2021) offer ways to learn to segment novel classes based on…”
GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, methodology): “For example, few-shot learning [21, 43, 50, 55, 70, 77, 84] and active learning [8, 62, 66, 67, 83] are proposed to perform segmentation with as few pixel-wise labels as possible.”
Novel Class Discovery in Semantic Segmentation (2021, methodology): “Following [24], we split the 80 foreground classes into 4 folds, and thus each fold contains 20 different classes, which are viewed as novel classes.”
Few-Shot Segmentation via Cycle-Consistent Transformer (2021, methodology): “Method Backbone 1-shot 5-shot 20 20(1) 20(2) 20(3) Mean 20 20(1) 20(2) 20(3) Mean FWB [20] Res-101 19.”
Prior Guided Feature Enrichment Network for Few-Shot Segmentation (2020, positive): “The prior masks provide the bias-free prior information from high-level features for both seen and unseen data during the evaluation, while masks produced by learnable feature maps (e.g., [24], [40], [50]) are affected by parameter learning during training.”
Visual Prompting via Image Inpainting (2022, methodology): “FWB [36] and CyCTR [59] outperform Visual Prompting, mainly because they pretrain on a large tagged base classes dataset and utilize architectures that are speciﬁc to image segmentation.”
Part-aware Prototype Network for Few-shot Semantic Segmentation (2020, methodology): “5M FWB[20] x RN101 51.”
Prototype-based Incremental Few-Shot Segmentation (2020, methodology): “Following FSS works [31, 42, 56], we consider 15 and 60 of the classes as Base ( C 0 ) and 5 and 20 as New ( C t \C 0 ), for VOC and COCO respectively.”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, methodology): “4, our model, which is trained without any data augmentation methods with 18 times smaller number of trainable parameters compared to [4] (2.6M vs. 46.7M), performs robustly in presence of large domain gaps between COCO-20i and PASCAL-5i, surpassing [4] by 1.0%p in 5-shot setting, and further improv”
Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, methodology): “Following [34, 25, 43], we use the mean Intersection over Union (mIoU) and binary Intersection over Union (FB-IoU) as the performance evaluation metrics.”
Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (2021, methodology): “Following FWB [20], the overall 80 classes from MSCOCO [17] are evenly divided into four splits with the same cross-validation strategy.”
Prototype Mixture Models for Few-shot Semantic Segmentation (2020, methodology): “Given a few examples, termed support images, and the related segmentation masks [13], few-shot segmentation aims to segment the query images based on a feature representation learned on training images.”
Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need? (2020, background): “Few-shot segmentation, which has gained popularity recently [10, 17, 19, 23, 25, 33, 36, 37, 38, 39, 41, 42], is no exception.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, neutral): “0 FWB [262] ICCV’19 ResNet-101 Metric 56.”
Few-Shot Semantic Segmentation with Democratic Attention Networks (2020, methodology): “To obtain this guidance, [41, 36, 22] adopted global average pooling by squeezing the support feature of the support image into a vector, and based the segmentation on a specific metric, e.g., cosine similarity, between the global vector and the feature map of the query image.”