Conditional Generative Adversarial Network for Structured Domain Adaptation

By Weixiang Hong, Zhenzhen Wang, Ming Yang, Junsong Yuan. 2018

301 citations recorded by the source corpus.

In recent years, deep neural nets have triumphed over many computer vision problems, including semantic segmentation, which is a critical task in emerging autonomous driving and medical image diagnostics applications. In general, training deep neural nets requires a humongous amount of labeled data, which is laborious and costly to collect and annotate. Recent advances in computer graphics shed light on utilizing photo-realistic synthetic data with computer generated annotations to train neural nets. Nevertheless, the domain mismatch between real images and synthetic ones is the major challenge against harnessing the generated data and labels. In this paper, we propose a principled way to conduct structured domain adaption for semantic segmentation, i.e., integrating GAN into the FCN framework to mitigate the gap between source and target domains. Specifically, we learn a conditional generator to transform features of synthetic images to real-image like features, and a discriminator to distinguish them. For each training batch, the conditional generator and the discriminator compete against each other so that the generator learns to produce real-image like features to fool the discriminator; afterwards, the FCN parameters are updated to accommodate the changes of GAN. In experiments, without using labels of real image data, our method significantly outperforms the baselines as well as state-of-the-art methods by 12% ~ 20% mean IoU on the Cityscapes dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

Domain Adaptation for Traffic Density Estimation (2021, background): “More recently, the work proposed in (Hong et al., 2018) employs a residual network and adversarial training to make the source feature maps closer to the target ones.”
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation (2018, methodology): “[16] uses a residual net to make the source feature maps similar to target’s ones us-ing adversarial training, the feature maps being then used for segmentation task.”
Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization (2020, methodology): “images from the target domain can be utilized to guide the optimization procedure. The general idea of domain adaption is to align the source domain and target domain distributions in the input level [3,19], semantic feature level [9,34], or output space level [5,39,40,43,45]. Most methods adopt Gen”
ACE: Adapting to Changing Environments for Semantic Segmentation (2019, unknown): “; or implicitly estimated to make features domain-invariant using adversarial loss functions such as reversed gradient [8, 9], domain confusion [57], or Generative Adversarial Network [58, 16, 17, 45, 18], etc .”
Synthetic Data for Deep Learning (2019, background): “[252] provide one of the most direct and most promising applications of feature-level synthetic-to-real domain adaptation.”
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, background): “A major class of adaptation approaches, including [5,12,13,19,28,31,34,36,43,46], involves adversarial confusion or feature alignment between domains.”
Small Sample Learning in Big Data Era (2018, background): “, 2017), and semantic segmentation (Hong et al., 2018).”
Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic Segmentation (2019, background): “In particular, most of the domain adaptation methods in semantic segmentation depend on adversarial training aiming to minimize the domain discrepancy through domain confusion [15, 14, 44, 41, 16, 52].”
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (2021, background): “As suggested by the theoretical analysis [3], domain alignment methods focus on reducing the distribution mismatch by optimizing some divergence [30, 35] or adopting adversarial training [20, 40] at either the image level [1, 12, 19, 26, 49, 63, 68], the intermediate feature level [6, 27, 48, 58] or”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, methodology): “To remedy this issue, various domain alignment strategies are proposed in [8, 2, 9–11].”
Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation (2019, background): “To address this issue, one promising approach is domain adaptation [1, 45, 15, 34, 36, 27, 33, 40, 47, 13] to reduce the domain shift and learn a shared discriminative model for both domains.”
DLOW: Domain Flow for Adaptation and Generalization (2018, background): “Recently, there is an increasing interest to apply domain adaptation techniques for semantic segmentation from synthetic data to the real scenario [22, 21, 7, 67, 40, 25, 11, 46, 51, 53, 23, 47, 62, 54, 43, 50, 52, 66, 5].”
Source-Free Domain Adaptation for Semantic Segmentation (2021, methodology): “Inspired by image-to-image translation [21, 54], a category of UDA methods has been proposed to generate target images conditioned on source data [19, 18].”