Semantic Segmentation using Adversarial Networks

By Pauline Luc, Camille Couprie, Soumith Chintala, Jakob Verbeek. 2016

756 citations recorded by the source corpus.

Adversarial training has been shown to produce state of the art results for generative image modeling. In this paper we propose an adversarial training approach to train semantic segmentation models. We train a convolutional semantic segmentation network along with an adversarial network that discriminates segmentation maps coming either from the ground truth or from the segmentation network. The motivation for our approach is that it can detect and correct higher-order inconsistencies between ground truth segmentation maps and the ones produced by the segmentation net. Our experiments show that our adversarial training approach leads to improved accuracy on the Stanford Background and PASCAL VOC 2012 datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Generative Adversarial Networks (2021, neutral): “By providing a thorough overview of GANs, we hope to make this complex and challenging topic more accessible to researchers and practitioners in the field, and inspire new innovations and applications.”
Learning Adversarially Fair and Transferable Representations (2018, methodology): “, 2016; Odena, 2016) and segmentation (Luc et al., 2016), although similar concepts had previously been proposed for unsupervised and supervised learning (Schmidhuber, 1992; Gutmann & Hyvärinen, 2010).”
Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “A large group of the existing UDA algorithms for image segmentation use adversarial learning for extracting domaininvariant features (Luc et al. 2016; Bousmalis et al. 2017; Hoffman et al. 2018b; Murez et al. 2018; Saito et al. 2018; Sankaranarayanan et al. 2018; Dhouib, Redko, and Lartizien 2020).”
Image Segmentation Using Deep Learning: A Survey (2020, neutral): “[100] proposed an adversarial training approach for semantic segmentation in which they trained a convolu-”
Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network (2018, methodology): “Apart from synthesizing data, several studies [20, 23, 36, 35] use adversarial learning as an extra supervision on the segmentation or detection networks.”
Semi-Supervised Semantic Segmentation With High- and Low-Level Consistency (2019, methodology): “[24] optimize the original GAN loss to encourage predicted segmentation maps to be similar to the ground-truth maps and show that it improves the performance in a fully-supervised setting.”
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort (2021, background): “Semi-Supervised Learning: Given a large set of unlabeled images and a small set of annotated images, semisupervised approaches [37, 47, 22, 40, 27] aim to learn better segmentation networks than with supervised data alone.”
Deep Learning for Cardiac Image Segmentation: A Review (2019, methodology): “In this way, the segmentation network is encouraged to produce more anatomically plausible segmentation maps (65, 66).”
Watch Your Up-Convolution: CNN Based Generative Deep Neural Networks Are Failing to Reproduce Spectral Distributions (2020, background): “…images [29, 6], image-to-image [45, 26, 61, 9, 42, 30] and text-to-image translations [48, 11, 58, 59], style transfer [27, 60, 61, 25], image inpainting [45, 54, 33, 26, 56], transfer learning [5, 10, 15] or even for training semantic segmentation tasks [35, 53], just to name a few.”
Generative Adversarial Networks for Noise Reduction in Low-Dose CT (2017, methodology): “Previously proposed GAN methods also used the input to G as input to D [31], [40].”
A Review of Deep-Learning-Based Medical Image Segmentation Methods (2021, neutral): “[52] trained a convolutional semantic segmentation network and an adversarial network.”
Adaptive Affinity Field for Semantic Segmentation (2018, background): “Unlike CRF and GAN that attempt to model the structural regularity based on the input/prediction appearance, our approach operates on the local affinity fields of prediction as well as ground-truth, requiring them to match in the properties derived from all the neighbourhoods instead of individual p”
A review of medical image data augmentation techniques for deep learning applications (2021, background): “Ideally, an equilibrium state is eventually reached, where the generator can approximate data from the target data distribution and the discriminator predicts ‘real’ or ‘generated’ for its input data with 50% probability.(129,130) Realistic synthetic data can be generated by the generator via sampli”
Deep learning in cancer diagnosis, prognosis and treatment selection (2021, background): “This ‘ adversarial ’ mechanism forces the generator to be as accurate as possible in localising objects so that the discriminator cannot recognise the difference be-tween predicted and ground-truth class labels [84].”
Improving Consistency and Correctness of Sequence Inpainting using Semantically Guided Generative Adversarial Network (2017, background): “Another contemporary practice is to condition a GAN on an auxiliary image, specially for the task of image-to-image translation [16, 7], style transfer [52, 6, 17], video/sequence generation [29, 43], image denoising [47], real time texture synthesis [26], image super resolution [25], semantic inpai”
Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images (2018, background): “According to [69, 70] higher order potentials have also been useful for image segmentation.”
Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation (2017, methodology): “Such approaches have been used successfully in natural image super-resolution (SR) [26] and segmentation [30] tasks.”
Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation (2019, background): “, 2017) and semantic segmentation (Luc et al., 2016), (Souly et al.”