Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features

By Xiang Wang, Shaodi You, Xi Li, Huimin Ma. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

320 citations recorded by the source corpus.

Weakly-supervised semantic segmentation under image tags supervision is a challenging task as it directly associates high-level semantic to low-level appearance. To bridge this gap, in this paper, we propose an iterative bottom-up and top-down framework which alternatively expands object regions and optimizes segmentation network. We start from initial localization produced by classification networks. While classification networks are only responsive to small and coarse discriminative object regions, we argue that, these regions contain significant common features about objects. So in the bottom-up step, we mine common object features from the initial localization and expand object regions with the mined features. To supplement non-discriminative regions, saliency maps are then considered under Bayesian framework to refine the object regions. Then in the top-down step, the refined object regions are used as supervision to train the segmentation network and to predict object masks. These object masks provide more accurate localization and contain more regions of object. Further, we take these object masks as initial localization and mine common object features from them. These processes are conducted iteratively to progressively produce fine object masks and optimize segmentation networks. Experimental results on Pascal VOC 2012 dataset demonstrate that the proposed method outperforms previous state-of-the-art methods by a large margin.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/affe7e93c31abf7a28088e5b3a83ddf24bf07a3f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation (2021, methodology): “[6, 40, 47] combine saliency maps with class-specific attention cues to generate reliable pseudo-masks.”
Weakly-Supervised Salient Object Detection via Scribble Annotations (2020, methodology): “[37] presented an iterative bottom-up and top-down semantic segmentation framework to alternatingly expand object regions and optimize segmentation network with image tag supervision.”
Identity-Guided Human Semantic Parsing for Person Re-Identification (2020, background): “semantic segmentation [16, 47, 19, 12, 11, 33, 44, 62], which aim to locate objects like person, horse or dog at pixel-level with image-level supervision.”
Single-Stage Semantic Segmentation From Image Labels (2020, background): “[54] iteratively train a network with the seed-and-expand principle, refine the intermediate results with saliency maps [53], and provide them as supervision to the segmentation network.”
Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification From the Bottom Up (2019, methodology): “The method in [44] proposed an iterative bottom-up and topdown framework to expand object regions and optimize segmentation network iteratively.”
C-MIDN: Coupled Multiple Instance Detection Network With Segmentation Guidance for Weakly Supervised Object Detection (2019, background): “Weakly Supervised Semantic Segmentation A signiﬁ-cant advance of performance of WSSS has been witnessed in last several years [44, 16, 38, 1, 29, 45, 41].”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, background): “Diverse solutions are explored, such as image-level operations [3], regions growing strategies[5], and feature-level enhancements[7].”
Learning Integral Objects With Intra-Class Discriminator for Weakly-Supervised Semantic Segmentation (2020, background): “, image-level labels [20, 40, 1, 38, 37, 15], scribbles [24, 36], and bounding boxes [7, 19, 34].”
Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation (2019, background): “[33] combined class-agnostic saliency maps with class-specific attention cues like us to obtain dense seed.”
FickleNet: Weakly and Semi-Supervised Semantic Image Segmentation Using Stochastic Inference (2019, unknown): “…53.8 AE_PSL CVPR ’17 [31] 10K 55.0 55.7 DCSP BMVC ’17 [2] 10K 58.6 59.2 MEFF CVPR ’18 [8] 10K - 55.6 GAIN CVPR ’18 [19] 10K 55.3 56.8 MCOF CVPR ’18 [30] 10K 56.2 57.6 AfﬁnityNet CVPR ’18 [1] 10K 58.4 60.5 DSRG CVPR ’18 [12] 10K 59.0 60.4 MDC CVPR ’18 [33] 10K 60.4 60.8 FickleNet (Ours) 10K 61.2…”
DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification (2022, background): “For example, one notable capability of CAM is the target localization of a model trained only with image labels; therefore, it prevails in weakly supervised tasks, such as segmentation [4, 13, 16, 43] and detection [38,45,50], or even knowledge distillation [39].”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, background): “[21], DCSP [8], TPL [27], GAIN [31], DSRG [23], MCOF [48], AffinityNet [2], RDC [52], SeeNet [22], OAA [24], ICD [17], BES [9], Fan et al.”
CycleSegNet: Object Co-Segmentation With Cycle Refinement and Region Correspondence (2021, unknown): “[40] proposes to use saliency maps to iterative refine weakly segmentation results.”
Salient Object Detection in the Deep Learning Era: An In-Depth Survey (2019, unknown): “In computer vision, sampled applications of SOD include image understanding [1], [2], image cap-tioning [3]–[5], object detection [6], [7], un-supervised video object segmentation [8], [9], semantic segmentation [10]– [12], person re-identiﬁcation [13], [14], etc .”