Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

By Qi Yao, Xiaojin Gong. 2019

85 citations recorded by the source corpus.

Weakly supervised semantic segmentation (WSSS) using only image-level labels can greatly reduce the annotation cost and therefore has attracted considerable research interest. However, its performance is still inferior to the fully supervised counterparts. To mitigate the performance gap, we propose a saliency guided self-attention network (SGAN) to address the WSSS problem. The introduced self-attention mechanism is able to capture rich and extensive contextual information but may mis-spread attentions to unexpected regions. In order to enable this mechanism to work effectively under weak supervision, we integrate class-agnostic saliency priors into the self-attention mechanism and utilize class-specific attention cues as an additional supervision for SGAN. Our SGAN is able to produce dense and accurate localization cues so that the segmentation performance is boosted. Moreover, by simply replacing the additional supervisions with partially labeled ground-truth, SGAN works effectively for semi-supervised semantic segmentation as well. Experiments on the PASCAL VOC 2012 and COCO datasets show that our approach outperforms all other state-of-the-art methods in both weakly and semi-supervised settings.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sungmin Cha
2. Taesup Moon
3. Beomyoung Kim
4. Wayne Zhang
5. Junmo Kim
6. Jongwuk Lee
7. Zhanghui Kuang
8. Hyunjung Shim

In Computer Vision and Pattern Recognition

1. Sungmin Cha
2. Taesup Moon
3. Beomyoung Kim
4. Y. Yoo
5. Wayne Zhang
6. Junmo Kim
7. Jongwuk Lee
8. Zhanghui Kuang

Citations in the corpus

• SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning (2021, background): “For example, the weakly-supervised semantic segmentation approaches [29, 31, 14] generate pseudo-labels filtering out the background regions using saliency map, and recent data augmentations [17, 16] also exploit the saliency information to find the optimal mixing of mask.”
• Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation (2021, methodology): “SGAN [47] has quite a similar confusion ratio with ours, but our method captures the target class much accurately in terms of IoU.”
• Pseudo-mask Matters in Weakly-supervised Semantic Segmentation (2021, methodology): “Besides the unsupervised contour learning method, other works introduce saliency detection models trained from extra dataset to refine CAMs [37, 38, 11].”
• Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation (2021, background): “More recently, RRM (Zhang et al. 2020) proposed a fully end-to-end network for joint training of classification and segmentation, and SGAN (Yao and Gong 2020) proposed a self-attention network guided by saliency priors that can produce dense and accurate localization maps from rich contextual inform”