Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation

By Seungho Lee, Minhyun Lee, Jongwuk Lee, Hyunjung Shim. 2021

276 citations recorded by the source corpus.

Existing studies in weakly-supervised semantic segmentation (WSSS) using image-level weak supervision have several limitations: sparse object coverage, inaccurate object boundaries, and co-occurring pixels from non-target objects. To overcome these challenges, we propose a novel framework, namely Explicit Pseudo-pixel Supervision (EPS), which learns from pixel-level feedback by combining two weak supervisions; the image-level label provides the object identity via the localization map and the saliency map from the off-the-shelf saliency detection model offers rich boundaries. We devise a joint training strategy to fully utilize the complementary relationship between both information. Our method can obtain accurate object boundaries and discard co-occurring pixels, thereby significantly improving the quality of pseudo-masks. Experimental results show that the proposed method remarkably outperforms existing methods by resolving key challenges of WSSS and achieves the new state-of-the-art performance on both PASCAL VOC 2012 and MS COCO 2014 datasets. The code is available at https://github.com/halbielee/EPS.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Barbara Caputo
2. Fabio Cermelli
3. Kate Saenko
4. Ping Hu
5. Marco Ciccone
6. Dario Fontanel
7. Berk Çalli
8. A. Tavera

In Computer Vision and Pattern Recognition

1. Barbara Caputo
2. Kate Saenko
3. Luc Van Gool
4. Trevor Darrell
5. Fabio Cermelli
6. Dacheng Tao
7. Ping Hu
8. Kristen Grauman

Citations in the corpus

• ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes (2021, methodology): “For CAM generation, we incorporated the current state-of-the-art Puzzle-Cam [27] and EPS [32] methods that enhances CAM object coverage by matching the CAM of an entire image with those produced for the non-overlapping tiles of the image.”
• Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “Directly applying existing weakly-supervised methods to incremental segmentation would require to (i) extract pixel-wise pseudo-supervision offline using a weakly supervised approach [3, 5, 36, 59, 63] and (ii) update the segmentation network resorting to an incremental learning technique [8,21,43].”
• Multi-class Token Transformer for Weakly Supervised Semantic Segmentation (2022, unknown): “More specifically, we used the pre-trained [33] ResNet101 I+S 66.2 66.9 EDAM (CVPR21) [42] ResNet101 I+S 70.9 70.6 EPS (CVPR21) [23] ResNet101 ResNet38 I 62.7 64.3 SEAM (CVPR20) [39] ResNet38 I 64.5 65.7 BES (ECCV20) [5] ResNet101 I 65.7 66.6 CONTA (NeurIPS20) [48] ResNet38 I 66.1 66.7 AdvCAM…”