Webly Supervised Semantic Segmentation

By Bin Jin, Maria V. Ortiz Segovia, Sabine Süsstrunk. 2017

91 citations recorded by the source corpus.

We propose a weakly supervised semantic segmentation algorithm that uses image tags for supervision. We apply the tags in queries to collect three sets of web images, which encode the clean foregrounds, the common backgrounds, and realistic scenes of the classes. We introduce a novel three-stage training pipeline to progressively learn semantic segmentation models. We first train and refine a class-specific shallow neural network to obtain segmentation masks for each class. The shallow neural networks of all classes are then assembled into one deep convolutional neural network for end-to-end training and testing. Experiments show that our method notably outperforms previous state-of-the-art weakly supervised semantic segmentation approaches on the PASCAL VOC 2012 segmentation benchmark. We further apply the class-specific shallow neural networks to object segmentation and obtain excellent results.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Extreme Cut: From Extreme Points to Object Segmentation (2017, methodology): “Image-level labels [26], noisy web labels [1, 16] and scribble-level labels [20] are some of the supervisory signal that have been used to guide segmentation methods.”
RECALL: Replay-based Continual Learning in Semantic Segmentation (2021, methodology): “Recently, it was employed also in semantic segmentation to provide a plentiful source of both images [20, 38] and videos [19] with weak image-level class labels during training.”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, methodology): “, images from existing datasets [15, 51]); or (2) utilizing near-infinite yet noisy web-sourced image [18,53,54,70] or video [18,32,58] data (also referred as webly supervised semantic segmentation [24]).”
Self-Erasing Network for Integral Object Attention (2018, unknown): “ CVPR’17 10K weak - 52.8% 53.7% Oh et al. [23] CVPR’17 10K weak + sal 51.2% 55.7% 56.7% AE-PSL [34] CVPR’17 10K weak + sal - 55.0% 55.7% Hong et al. [9] CVPR’17 10K + video weak - 58.1% 58.7% WebS-i2 [14] CVPR’17 19K weak - 53.4% 55.3% DCSP-VGG16 [3] BMVC’17 10K weak + sal 56.5% 58.6% 59.2% DCSP-Res”
Webly Supervised Concept Expansion for General Purpose Vision Models (2022, methodology): “As web results improved, works used this data to build object detectors [11, 15, 46, 54, 71, 83], attribute detectors [21], image taggers [81], large vocabulary categorization models [24,56,85] and finegrained recognition models [39, 57], segmentation models [33, 70, 73], online dataset builders [45”
Bootstrapping the Performance of Webly Supervised Semantic Segmentation (2018, methodology): “The hope is that these extra data can be used to boost the performance, and indeed a number of papers [30, 11, 26] have previously explored this idea to improve results of weakly supervised methods.”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, methodology): “[38], AE-PSL [50], WebS-i2 [25], Hong et al.”
Weakly Supervised Segmentation with Maximum Bipartite Graph Matching (2020, background): “For example, weak supervisions with bounding boxes[8, 39], scribbles[32, 33], image labels[2, 3, 22, 27, 44, 46, 48, 53], and extreme points[4] annotations are widely studied in the literature.”