Adversarial Learning for Semi-supervised Semantic Segmentation

By Wei-Chih Hung, Yi-Hsuan Tsai, Yan-Ting Liou, Yen-Yu Lin, Ming-Hsuan Yang. 2018

619 citations recorded by the source corpus.

We propose a method for semi-supervised semantic segmentation using an adversarial network. While most existing discriminators are trained to classify input images as real or fake on the image level, we design a discriminator in a fully convolutional manner to differentiate the predicted probability maps from the ground truth segmentation distribution with the consideration of the spatial resolution. We show that the proposed discriminator can be used to improve semantic segmentation accuracy by coupling the adversarial loss with the standard cross entropy loss of the proposed model. In addition, the fully convolutional discriminator enables semi-supervised learning through discovering the trustworthy regions in predicted results of unlabeled images, thereby providing additional supervisory signals. In contrast to existing methods that utilize weakly-labeled images, our method leverages unlabeled images to enhance the segmentation model. Experimental results on the PASCAL VOC 2012 and Cityscapes datasets demonstrate the effectiveness of the proposed algorithm.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

An EM Framework for Online Incremental Learning of Semantic Segmentation (2021, neutral): “Re-labeling, also known as self-training [16, 25, 33], is a strategy to learn from partially labeled or unlabeled data.”
Image Segmentation Using Deep Learning: A Survey (2020, methodology): “[102] developed a framework for semi-supervised semantic segmentation using an adversarial network.”
A Survey on Curriculum Learning (2021, methodology): “An analogous idea is adopted in [38] by assigning binary selection on unlabeled data based on pretrained discriminator on labeled data in semi-supervised semantic segmentation task.”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, methodology): “We also benchmark our approach against several state-of-the-art SLL methods for segmentation that have code available: the mean teacher model with transformation-consistency (MT) [87, 53], the adversarial training-based method [37] (AdvSSL), and also the recently proposed Guided Collaborative Traini”
Unsupervised Domain Adaptation for Semantic Segmentation of Urban Scenes (2019, methodology): “Finally, the third term is based on a self-teaching framework inspired from [15], where the predicted segmentation is passed through the discriminator to obtain a conﬁdence map and then high conﬁdence regions are considered reliable and used as ground truth for self-teaching the network over them.”
Boundary-weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation (2019, background): “The networks SNet-s, SNet-t and D are designed to work in an adversarial fashion, which is derived from the idea of adversarial learning [24], [43], to overcome the problem of domain shifting and more importantly to exploit the information carried by datasets from source domain to deal with the prob”
Guided Collaborative Training for Pixel-wise Semi-Supervised Learning (2020, methodology): “Following previous works [19,33], we use DeepLab-v2 [9] with the ResNet-101 [17] backbone as the SupOnly baselines and as the task model in SSL methods.”
DACS: Domain Adaptation via Cross-domain Mixed Sampling (2020, methodology): “ annotate some data from the domain of interest and re-train (ﬁne-tune) the network on this new data. Additionally, Semi-Supervised Learning (SSL) methods for semantic segmentation have been proposed [4, 5, 6, 7, 8, 9], effectively training on a small amount of labelled data by relying on complement”
Extract Free Dense Labels from CLIP (2021, methodology): “Self-training has firstly become popular in the semisupervised classification task [33,29,35,43] and is also recently applied in the semi-supervised/zero-shot semantic segmentation settings [27,37,4,28,40,36,56,8].”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, methodology): “These works generally inherit similar learning rationales as SSL or UL for image classification, and adapt techniques such as consistency regularization [216], [217], [218], [219], self-training [202], [210], [220], [221], [222], [223], [224], GAN frameworks [225], [226], [227] in SSL, or contrastiv”
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation (2020, methodology): “Method 1/2 (732) 1/4 (366) 1/8 (183) 1/16 (92) AdvSemSeg (Hung et al., 2018) 65.”
Transformation-Consistent Self-Ensembling Model for Semisupervised Medical Image Segmentation (2019, methodology): “Note that the method [19] for medical image segmentation adopts a similar idea with the adversarial learning-based method [59].”
On Regularized Losses for Weakly-supervised CNN Segmentation (2018, background): “Note that this result is highly preliminary and detailed analysis of overﬁtting, generalization property and comparison to recent semi-supervised segmentation [34] with extra unlabeled images will be our future work.”
Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images (2020, methodology): “The first group of those methods mainly consider the generic setting of semi-supervised segmentation [3, 7–9, 11, 16, 18–20].”
Semi-Supervised Semantic Segmentation With High- and Low-Level Consistency (2019, methodology): “[15], and achieved an improvement of 6.”
ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning (2020, methodology): “For semantic segmentation, semi-supervised learning has been explored with techniques based on adversarial learning [15, 16], consistency regularization [6, 9, 17, 18], and pseudo-labelling [19, 20].”
Weakly-Supervised Salient Object Detection via Scribble Annotations (2020, methodology): “[12] randomly interleaved labeled and unlabeled data, and trained a network with an adversarial loss on the unlabeled data for semi-supervised semantic segmentation.”
Instance Adaptive Self-Training for Unsupervised Domain Adaptation (2020, methodology): “As shown in Table 8, we have significantly better performance than [11] and [12].”
ST++: Make Self-trainingWork Better for Semi-supervised Semantic Segmentation (2021, methodology): “Motivated by the tremendous progress in SSL, recent works on semi-supervised semantic segmentation are evolving from GANs-based methods [17, 44, 21, 32] to integrating the advanced SSL methods into the segmentation scenario, such as enforcing consistent predictions of the same unlabeled image but un”
Adversarial Learning and Self-Teaching Techniques for Domain Adaptation in Semantic Segmentation (2019, methodology): “The third loss function starts from the work of Hung et al. [5].”