CrDoCo: Pixel-Level Domain Transfer With Cross-Domain Consistency

By Yun-Chun Chen, Yen-Yu Lin, Ming-Hsuan Yang, Jia-Bin Huang. 2019

300 citations recorded by the source corpus.

Unsupervised domain adaptation algorithms aim to transfer the knowledge learned from one domain to another (e.g., synthetic to real images). The adapted representations often do not capture pixel-level domain shifts that are crucial for dense prediction tasks (e.g., semantic segmentation). In this paper, we present a novel pixel-wise adversarial domain adaptation algorithm. By leveraging image-to-image translation methods for data augmentation, our key insight is that while the translated images between domains may differ in styles, their predictions for the task should be consistent. We exploit this property and introduce a cross-domain consistency loss that enforces our adapted model to produce consistent predictions. Through extensive experimental results, we show that our method compares favorably against the state-of-the-art on a wide variety of unsupervised domain adaptation tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Knowledge Distillation: A Survey (2020, methodology): “In the ﬁrst category, an adversarial generator is trained to generate synthetic data, which is either directly used as the training dataset (Chen et al. 2019a; Ye et al. 2020) or used to augment the training dataset (Liu et al. 2018), shown in Fig.”
DACS: Domain Adaptation via Cross-domain Mixed Sampling (2020, methodology): “ has instead been used to complement minimization of distribution discrepancies on an image level, with consistency criteria aimed at enforcing consistent predictions over image-to-image translations [26, 27]. For semantic segmentation, existing research in SSL has seen success in enforcing consiste”
Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition (2019, methodology): “It has been successfully applied to unsupervised domain adaptation [16, 53] and later extended to pixel-wise adaptation [6, 24, 10] and multi-source adaptation [25].”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, background): “For adversarial learning, numerous works focus on reducing the distribution misalignment in the image level [12, 13, 10, 14, 15, 5, 16], feature level [17, 8, 9], and output level [2–4 , 18].”
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search (2020, background): “Among the design choices, 2D transposed convolution is one of the popular choices for various dense prediction tasks, including semantic segmentation [54], optical flow estimation [21], depth prediction [15], and image restoration [49, 73] and synthesis [59] problems.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “These models typically learn a real-to-real [212], [213], [214] or synthetic-to-real [210], [211] mapping, rendering the style of the labeled source to the unlabeled target domain, thus offering synthetic training data with pseudo labels.”
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (2021, background): “As suggested by the theoretical analysis [3], domain alignment methods focus on reducing the distribution mismatch by optimizing some divergence [31, 36] or adopting adversarial training [21, 41] at either the image level [1, 12, 13, 20, 27, 50, 64], the intermediate feature level [6, 28, 49, 59] or”
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation (2020, background): “…been proposed to apply adversarial training to align the source and target distributions on the feature-level (Tzeng et al., 2017; Chen et al., 2018; Hsu et al., 2020) or on the pixel-level (Tsai et al., 2018; Hoffman et al., 2018; Bous-malis et al., 2017; Chen et al., 2019b; Lee et al., 2018).”
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning (2020, methodology): “The assumption of cyclical structure has been used in various works [83, 84, 85].”
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations (2019, methodology): “In addition, I2I translation can be applied to synthesize images for domain adaptation (Bousmalis et al. 2017; Chen et al. 2019; Hoffman et al. 2018; Murez et al. 2018; Shrivastava et al. 2017).”
Adversarial Learning and Self-Teaching Techniques for Domain Adaptation in Semantic Segmentation (2019, background): “In [56] the region growing problem is deﬁned as a Markov Decision Process.”
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (2020, methodology): “Some methods such as [35], [88] distill the knowledge from source domain to target domain based on adversarial learning [65] and image translation [97], [222], [224].”
PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training (2021, background): “Beyond cycle-consistency, [12] enforces cross-domain consistent predictions in the context of image-to-image translation for UDA for using two image-level adversarial losses and two feature-level adversarial losses.”
Phase Consistent Ecological Domain Adaptation (2020, methodology): “In addition to cycle-consistency, [24, 8] propose using the segmentation network on the target domain to encourage better semantic consistency.”