Bootstrapping the Performance of Webly Supervised Semantic Segmentation

By Tong Shen, Guosheng Lin, Chunhua Shen, Ian David Reid. 2018

72 citations recorded by the source corpus.

Fully supervised methods for semantic segmentation require pixel-level class masks to train, the creation of which is expensive in terms of manual labour and time. In this work, we focus on weak supervision, developing a method for training a high-quality pixel-level classifier for semantic segmentation, using only image-level class labels as the provided ground-truth. Our method is formulated as a two-stage approach in which we first aim to create accurate pixel-level masks for the training images via a bootstrapping process, and then use these now-accurately segmented images as a proxy ground-truth in a more standard supervised setting. The key driver for our work is that in the target dataset we typically have reliable ground-truth image-level labels, while data crawled from the web may have unreliable labels, but can be filtered to comprise only easy images to segment, therefore having reliable boundaries. These two forms of information are complementary and we use this observation to build a novel bi-directional transfer learning framework. This framework transfers knowledge between two domains, target domain and web domain, bootstrapping the performance of weakly supervised semantic segmentation. Conducting experiments on the popular benchmark dataset PASCAL VOC 2012 based on both a VGG16 network and on ResNet50, we reach state-of-the-art performance with scores of 60.2% IoU and 63.9% IoU respectively1.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

RECALL: Replay-based Continual Learning in Semantic Segmentation (2021, methodology): “Recently, it was employed also in semantic segmentation to provide a plentiful source of both images [20, 38] and videos [19] with weak image-level class labels during training.”
A Mutual Bootstrapping Model for Automated Skin Lesion Segmentation and Classification (2019, background): “Shen et al. [32] proposed a bi-directional transfer learning framework to generate high quality object localization masks.”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, methodology): “, images from existing datasets [15, 51]); or (2) utilizing near-infinite yet noisy web-sourced image [18,53,54,70] or video [18,32,58] data (also referred as webly supervised semantic segmentation [24]).”
PreDet: Large-scale weakly supervised pre-training for detection (2021, background): “Learning detection and semantic segmentation models [9, 8, 53, 57, 21, 46, 47] directly from web-scale data in a weakly-supervised manner has also shown some promise.”
Weakly Supervised Segmentation with Maximum Bipartite Graph Matching (2020, methodology): “In order to handle the data deficiency problem in the image segmentation task, researchers have explored various types of weak supervision, such as bounding boxes [8, 39], scribbles [32, 33], and image labels[2, 3, 12, 22, 27, 42, 44, 46–48, 53, 66, 67, 67].”
Webly Supervised Concept Expansion for General Purpose Vision Models (2022, methodology): “[70] Tong Shen, Guosheng Lin, Chunhua Shen, and Ian D.”