Exploiting Saliency for Object Segmentation from Image Level Labels

By Seong Joon Oh, Rodrigo Benenson, Anna Khoreva, Zeynep Akata, Mario Fritz, Bernt Schiele. 2017

209 citations recorded by the source corpus.

There have been remarkable improvements in the semantic labelling task in the recent years. However, the state of the art methods rely on large-scale pixel-level annotations. This paper studies the problem of training a pixel-wise semantic labeller network from image-level annotations of the present object classes. Recently, it has been shown that high quality seeds indicating discriminative object regions can be obtained from image-level labels. Without additional information, obtaining the full extent of the object is an inherently ill-posed problem due to co-occurrences. We propose using a saliency model as additional information and hereby exploit prior knowledge on the object extent and image statistics. We show how to combine both information sources in order to recover 80% of the fully supervised performance – which is the new state of the art in weakly supervised training for pixel-wise semantic labelling.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SmoothGrad: removing noise by adding noise (2017, methodology): “To measure spatial coherence, one might use existing databases of image segmentations, and we are already making progress (Oh et al., 2017; Selvaraju et al., 2016).”
Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “Most image-based weakly supervised approaches [2, 3, 10, 31, 34, 35, 48, 59] use a two-stage procedure: (i) they generate pixel-wise pseudo-labels and then (ii) use them for training a segmentation backbone.”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, methodology): “Note that saliency priors, used by many works such as [10, 31, 32] on Pascal VOC, are not suitable for “stuff” classes as popular saliency datasets [34–36] only consider “things” to be salient.”
Single-Stage Semantic Segmentation From Image Labels (2020, methodology): “Nevertheless, we surpass a number of multi-stage methods that use additional data and saliency supervision [5, 24, 28, 37, 56].”
Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (2018, methodology): “Among the approaches, CCNN [21], MIL-seg [23], STC [35], GuidedSeg [19], and TransferNet [10] use more images for training (700K, 700K, 50K, 20K and 70K, respectively).”
Consistent Structural Relation Learning for Zero-Shot Segmentation (2020, background): “Weakly supervised segmentation [7, 8] targets on learning segmentation models using lower-quality annotations such as image-level labels [9, 10], bounding boxes [11, 12] and scribbles [13, 14], which can be obtained more efﬁciently compared to pixel-wise masks.”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, unknown): “[38], AE-PSL [50], WebS-i2 [25], Hong et al.”
Effective Use of Synthetic Data for Urban Scene Semantic Segmentation (2018, background): “Weakly-supervised semantic segmentation aims to leverage a weaker form of annotation, such as image tags [23, 30, 32, 34, 40, 41, 49, 48, 29], bounding boxes [8, 21], scribbles [2] and object size statistics [31], which are cheaper to obtain.”
Attention-Based Dropout Layer for Weakly Supervised Object Localization (2019, methodology): “Existing approaches mine and track discriminative features of each class for object detection [45, 36, 37, 9, 45, 25, 21, 41, 19, 2, 39, 15, 63, 7, 5, 4, 48, 14, 65, 32, 31, 58, 62, 8, 6] and segmentation [33, 29, 18, 16, 24, 52, 50].”
Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations (2019, background): “In particular, image-level class labels have been widely used as weak labels since they require minimal or no effort for annotation [1, 12, 21, 22, 38, 42, 43, 48, 53].”
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation (2020, methodology): “…including partial image/feature erasing (Hou et al., 2018; Wei et al., 2017; Li et al., 2018), using an additional saliency estimation model (Oh et al., 2017; Huang et al., 2018; Wei et al., 2018), utilizing pixel similarity to propagate the initial score map (Ahn & Kwak, 2018; Wang et…”
Semantic Projection Network for Zero- and Few-Label Semantic Segmentation (2019, methodology): “As weak supervision, prior works use image-level annotation [34, 32, 31], point [5], scribble [25] and bounding box [20] annotations.”
Bootstrapping the Performance of Webly Supervised Semantic Segmentation (2018, unknown): “[21] 55.”
Surface defect saliency of magnetic tile (2018, background): “It is also worth noting that the recent progress [17]–[19] in the saliency detection are mostly relies on deep learning.”
Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation (2021, background): “ies [20, 25, 28, 35], we observe that some background classes frequently appear with target objects in PASCAL VOC 2012.”
Discovering Class-Specific Pixels for Weakly-Supervised Semantic Segmentation (2017, methodology): “Likewise the performance of [20] deteriorates significantly (from 55.”
Context-aware Feature Generation For Zero-shot Semantic Segmentation (2020, background): “To reduce the annotation pressure, leveraging weak annotations like image-level [34, 35, 47], box-level [18, 36], or scribble-level [24] annotations for semantic segmentation recently gained the interest of researchers.”
Learning Pixel-Level Semantic Affinity with Image-Level Supervision for Weakly Supervised Semantic Segmentation (2018, methodology): “Among various types of weak annotations for semantic segmentation, image-level class labels have been widely used [11, 14, 17, 26, 29, 30, 37] since they are already given in existing large-scale image datasets ( e.g. , ImageNet [7]) or automatically annotated for image retrieval results by search…”
Self-Erasing Network for Integral Object Attention (2018, unknown): “C [16] ECCV’16 10K weak 44.3% 50.7% 51.7% AugFeed [27] ECCV’16 10K weak + bbox 50.4% 54.3% 55.5% STC [35] PAMI’16 10K weak + sal - 49.8% 51.2% Roy et al. [28] CVPR’17 10K weak - 52.8% 53.7% Oh et al. [23] CVPR’17 10K weak + sal 51.2% 55.7% 56.7% AE-PSL [34] CVPR’17 10K weak + sal - 55.0% 55.7% Hong ”