Discovering Class-Specific Pixels for Weakly-Supervised Semantic Segmentation

By Arslan Chaudhry, Puneet Kumar Dokania, Philip H. S. Torr. 2017

124 citations recorded by the source corpus.

We propose an approach to discover class-specific pixels for the weakly-supervised semantic segmentation task. We show that properly combining saliency and attention maps allows us to obtain reliable cues capable of significantly boosting the performance. First, we propose a simple yet powerful hierarchical approach to discover the class-agnostic salient regions, obtained using a salient object detector, which otherwise would be ignored. Second, we use fully convolutional attention maps to reliably localize the class-specific regions in a given image. We combine these two cues to discover class-specific pixels which are then used as an approximate ground truth for training a CNN. While solving the weakly supervised semantic segmentation task, we ensure that the image-level classification task is also solved in order to enforce the CNN to assign at least one pixel to each object present in the image. Experimentally, on the PASCAL VOC12 val and test sets, we obtain the mIoU of 60.8% and 61.9%, achieving the performance gains of 5.1% and 5.2% compared to the published state-of-the-art results. The code is made publicly available.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Incremental Learning in Semantic Segmentation from Image Labels (2021, background): “Previous works focused on improving the pseudo-labels through multiple refinements steps [2, 3], additional losses [5, 10, 31, 34, 59, 63], or erasing techniques that force the CAM to expand and focus on non-discriminative parts of the image [12, 29, 64].”
Single-Stage Semantic Segmentation From Image Labels (2020, methodology): “[5] iteratively applied an off-the-shelf saliency detector [35] to a progressively erased image in order to accumulate the foreground mask.”
Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation (2020, unknown): “2 DCSP [4] ResNet101 √ 60.”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, unknown): “5 †DCSP [4] BMVC17 60.”
Adversarial Complementary Learning for Weakly Supervised Object Localization (2018, methodology): “Recently, some methods [17, 4] have already obtained localization maps like this, but we are the first to prove this convenient approach can generate same-quality localization maps with CAM, which is meaningful and contributes to embedding localization maps into complex networks.”
Weakly Supervised Segmentation with Maximum Bipartite Graph Matching (2020, methodology): “Although these methods adopt various additional information, such as saliency masks[5, 17, 18, 30], instance saliency map [11] and web images [46], our method still outperforms all previous results and achieves new state-of-the-art performance.”
Self-Erasing Network for Integral Object Attention (2018, methodology): “2% DCSP-ResNet101 [3] BMVC’17 10K weak + sal 59.”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, methodology): “We also train Deeplab-LargeFOV [94] based on ResNet [16] following the setting in [95].”
Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation (2019, background): “which are too sparse to effectively supervise a segmentation model, various techniques such as adversarial erasing [12], [17], [18], [19], attention accumulation [20], and region growing [13], [21], [22] have been developed to expand sparse object seeds.”
Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (2018, methodology): “AE-PSL [33] and DCSP [4] achieve the best performance among the baselines.”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, background): “Compared to bounding boxes [14, 26, 43], points [4], and scribbles [33, 46], image-level labels [29, 51, 21, 8, 23, 2, 52, 24] are the most widely used weak annotations due to their easy availability.”
Learning Integral Objects With Intra-Class Discriminator for Weakly-Supervised Semantic Segmentation (2020, unknown): “DCSP [2]BMVC17 I.”
Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation (2021, background): “[6, 40, 47] combine saliency maps with class-specific attention cues to generate reliable pseudo-masks.”
Weakly-Supervised Semantic Segmentation via Sub-Category Exploration (2020, unknown): “DCSP BMVC’17 [4] ResNet-101 60.”