Weakly-Supervised Semantic Segmentation via Sub-Category Exploration

By Yu-Ting Chang, Qiaosong Wang, Wei-Chih Hung, Robinson Piramuthu, Yi-Hsuan Tsai, Ming-Hsuan Yang. 2020

310 citations recorded by the source corpus.

Existing weakly-supervised semantic segmentation methods using image-level annotations typically rely on initial responses to locate object regions. However, such response maps generated by the classification network usually focus on discriminative object parts, due to the fact that the network does not need the entire object for optimizing the objective function. To enforce the network to pay attention to other parts of an object, we propose a simple yet effective approach that introduces a self-supervised task by exploiting the sub-category information. Specifically, we perform clustering on image features to generate pseudo sub-categories labels within each annotated parent class, and construct a sub-category objective to assign the network to a more challenging task. By iteratively clustering image features, the training process does not limit itself to the most discriminative object parts, hence improving the quality of the response maps. We conduct extensive analysis to validate the proposed method and show that our approach performs favorably against the state-of-the-art approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes (2021, methodology): “These methods typically utilize Class Activation Maps (CAM) [64] to select the most discriminative regions, which are later used as pixel-level supervision for segmentation networks [10,21,56].”
Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “Previous works focused on improving the pseudo-labels through multiple refinements steps [2, 3], additional losses [5, 10, 31, 34, 59, 63], or erasing techniques that force the CAM to expand and focus on non-discriminative parts of the image [12, 29, 64].”
GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, background): “Another line of related research is of weaklysupervised semantic segmentation [1,9,26,31,36,40,65,68, 80], which aims to learn semantic segmentation with only image-level object category supervision.”
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation (2022, methodology): “Prior works also exploited sub-categories [4] and cross-image semantics [13, 25, 33] to localize more accurate object regions.”
Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation (2021, background): “[5] proposes a self-supervised task to discover sub-categories to enforce the classifier to improve CAM.”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “To cope with the challenge, there have been various attempts in semi- and weakly-supervised segmentation approaches [6, 26, 39, 66, 72, 77, 88] which in turn effectively alleviated the data-hunger issue.”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, methodology): “For the segmentation network, following [7, 62, 18, 9], we adopt the DeepLab-v2 [10] framework.”
Pseudo-mask Matters in Weakly-supervised Semantic Segmentation (2021, methodology): “The methods include adding CRF and dilation [34], designing new losses [17], erasing high response area [13], keeping scale consistency via siamese network and correlation module [33], clustering sub-categories for classification [5].”