Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations

By Jiwoon Ahn, Sunghyun Cho, Suha Kwak. 2019

654 citations recorded by the source corpus.

This paper presents a novel approach for learning instance segmentation with image-level class labels as supervision. Our approach generates pseudo instance segmentation labels of training images, which are used to train a fully supervised model. For generating the pseudo labels, we first identify confident seed areas of object classes from attention maps of an image classification model, and propagate them to discover the entire instance areas with accurate boundaries. To this end, we propose IRNet, which estimates rough areas of individual instances and detects boundaries between different object classes. It thus enables to assign instance labels to the seeds and to propagate them within the boundaries so that the entire areas of instances can be estimated accurately. Furthermore, IRNet is trained with inter-pixel relations on the attention maps, thus no extra supervision is required. Our method with IRNet achieves an outstanding performance on the PASCAL VOC 2012 dataset, surpassing not only previous state-of-the-art trained with the same level of supervision, but also some of previous models relying on stronger supervision.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “Previous works focused on improving the pseudo-labels through multiple refinements steps [2, 3], additional losses [5, 10, 31, 34, 59, 63], or erasing techniques that force the CAM to expand and focus on non-discriminative parts of the image [12, 29, 64].”
Causal Intervention for Weakly-Supervised Semantic Segmentation (2020, methodology): “To this end, existing methods mainly focus on generating better seed areas [25, 57, 59, 58] and expanding these seed areas [1, 2, 20, 23, 55].”
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation (2020, background): “Although many weakly-supervised segmentation approaches (Ahn & Kwak, 2018; Ahn et al., 2019; Sun et al., 2020) aim at reﬁning CAM localization maps to segmentation masks, most of them have complicated post-processing steps, such as dense CRF (Kr ¨ ahenb ¨ uhl & Koltun, 2011), which increases the…”
Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation (2021, background): “[1, 2] implicitly exploit the boundary information by calculating affinities between pixels.”
Puzzle-CAM: Improved Localization Via Matching Partial And Full Features (2021, methodology): “IRNet [11] generates a transition matrix from the boundary activation map and extends the method to achieve weakly supervised instance segmentation and WSSS. SEAM [5] aims to reﬁne CAMs using a pixel correlation module that captures context appearance information for each pixel and alters the…”
Single-Stage Semantic Segmentation From Image Labels (2020, methodology): “5 w/ GCI for SG and use PAMR kernel [1, 2, 4, 8, 12, 24].”
Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorization (2020, neutral): “In addition to the human user, the “computer user” can also benefit from such methods, which can seed image segmentation techniques (Ahn, Cho, and Kwak 2019; Huang et al. 2018; Wang et al. 2019; Hoyer et al. 2019), or help focus generative image models, among other tasks.”
Pseudo-mask Matters in Weakly-supervised Semantic Segmentation (2021, background): “[2, 1] learn the contour of objects via an affinity network in inference stage.”
Unsupervised Semantic Segmentation by Distilling Feature Correspondences (2022, background): “CRF post-processing for reﬁning supervised and weakly supervised semantic segmentation predictions is ubiquitous in the literature (Lafferty et al., 2001; Chen et al., 2014; Long et al., 2015; Liu et al., 2020; Ahn et al., 2019).”
Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation (2020, methodology): “IRNet [1] generates a transition matrix from the boundary activation map and extends the method to weakly supervised instance segmentation.”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, neutral): “8 †IRN [1] CVPR19 63.”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, neutral): “[62], MCIS [44], IRN [1], FickleNet[30], SSDD [42], SEAM [49], SCE [7], CONTA [60].”
Weakly Supervised Segmentation with Maximum Bipartite Graph Matching (2020, methodology): “For example, Wei et al. [55] proposes to iteratively erase the discriminative regions and force the network to discover more object regions, Ahn et al. [2] utilize the inter-pixel relations to refine the object boundary.”