Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic Segmentation

By Jaehoon Choi, Taekyung Kim, Changick Kim. 2019

266 citations recorded by the source corpus.

Deep learning-based semantic segmentation methods have an intrinsic limitation that training a model requires a large amount of data with pixel-level annotations. To address this challenging issue, many researchers give attention to unsupervised domain adaptation for semantic segmentation. Unsupervised domain adaptation seeks to adapt the model trained on the source domain to the target domain. In this paper, we introduce a self-ensembling technique, one of the successful methods for domain adaptation in classification. However, applying self-ensembling to semantic segmentation is very difficult because heavily-tuned manual data augmentation used in self-ensembling is not useful to reduce the large domain gap in the semantic segmentation. To overcome this limitation, we propose a novel framework consisting of two components, which are complementary to each other. First, we present a data augmentation method based on Generative Adversarial Networks (GANs), which is computationally efficient and effective to facilitate domain alignment. Given those augmented images, we apply self-ensembling to enhance the performance of the segmentation network on the target domain. The proposed method outperforms state-of-the-art semantic segmentation methods on unsupervised domain adaptation benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, neutral): “We have compared our performance against the adversarial learning-based UDA methods: GIO-Ada (Chen et al. 2019b), ADVENT (Vu et al. 2019), AdaSegNet (Tsai et al. 2018), TGCF-DA+SE (Choi, Kim, and Kim 2019), PCEDA (Yang et al. 2020), and CyCADA (Hoffman et al. 2018a).”
Incremental Object Detection via Meta-Learning (2020, neutral): “This has found success in semi-supervised learning [50], [51], [52] and domain adaptation [53], [54], [55], [56] settings.”
DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation (2021, methodology): “In order to avoid training instabilities, pseudo-label prototypes [88] or consistency regularization [61,65] based on data augmentation [1,9,50] or domain-mixup [67, 91] are used.”
MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation (2022, methodology): “To increase the robustness of the self-training, consistency regularization [72,78,82] is often applied to ensure consistency over different data augmen-tations [1,9,19,63], different crops [33,42], multiple models [101,106,107], or domain-mixup [32–34,83,108].”
MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation (2021, unknown): “The general idea of UDA semantic segmentation methods is to perform domain alignment through adversarial learning [2, 6, 7, 9, 11, 20, 22, 26, 40, 41] or utilize self-training strategy on target samples to adapt the segmentation models [4, 17, 23, 28, 30, 35, 46, 48, 49, 50, 51].”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, background): “For adversarial learning, numerous works focus on reducing the distribution misalignment in the image level [12, 13, 10, 14, 15, 5, 16], feature level [17, 8, 9], and output level [2–4 , 18].”
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (2020, methodology): “To address the problem, several KD methods based on S-T frameworks [230, 231, 232, 276, 278, 279, 280, 281, 282] have been proposed recently.”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, unknown): “In order to regularize the training and to avoid pseudo-label drift, approaches such as confidence thresholding [43, 92], pseudo-label prototypes [39, 80, 81], and consistency regularization [50, 52, 57] based on data augmentation [1, 10, 44, 46], different context [35, 89], domain-mixup [20, 29, 39”
Deep semantic segmentation of natural and medical images: a review (2019, methodology): “Other works include GAN-based data augmentation for domain adaptation (Huang et al., 2018; Choi et al., 2019) and panoptic data augmentation (Liu et al.”
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort (2021, background): “In domain adaptation [41, 57, 51, 10], several works aimed at translating a labeled image dataset into another domain, in which image annotation is either expensive or missing entirely, by leveraging image-to-image translation techniques.”