Image to Image Translation for Domain Adaptation

By Zak Murez, Soheil Kolouri, David J. Kriegman, Ravi Ramamoorthi, Kyungnam Kim. 2017

561 citations recorded by the source corpus.

We propose a general framework for unsupervised domain adaptation, which allows deep neural networks trained on a source domain to be tested on a different target domain without requiring any training annotations in the target domain. This is achieved by adding extra networks and losses that help regularize the features extracted by the backbone encoder network. To this end we propose the novel use of the recently proposed unpaired image-to-image translation framework to constrain the features extracted by the encoder network. Specifically, we require that the features extracted are able to reconstruct the images in both domains. In addition we require that the distribution of features extracted from images in the two domains are indistinguishable. Many recent works can be seen as specific cases of our general framework. We apply our method for domain adaptation between MNIST, USPS, and SVHN datasets, and Amazon, Webcam and DSLR Office datasets in classification tasks, and also between GTA5 and Cityscapes datasets for a segmentation task. We demonstrate state of the art performance on each of these datasets.

Subfields: Artificial Intelligence, Cancer Research, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “A large group of the existing UDA algorithms for image segmentation use adversarial learning for extracting domaininvariant features (Luc et al. 2016; Bousmalis et al. 2017; Hoffman et al. 2018b; Murez et al. 2018; Saito et al. 2018; Sankaranarayanan et al. 2018; Dhouib, Redko, and Lartizien 2020).”
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation (2018, background): “Among some other approaches, [26] uses a combination of adversarial and generative techniques through multiple losses, [46] combines the generative approach for appearance adaptation and adversarial training for representation adaptation, and [45] propose a curriculum-style learning for UDA by…”
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort (2021, background): “In domain adaptation [41, 57, 51, 10], several works aimed at translating a labeled image dataset into another domain, in which image annotation is either expensive or missing entirely, by leveraging image-to-image translation techniques.”
Confidence Regularized Self-Training (2019, methodology): “To this end, a predominant stream of adversarial learning based UDA methods were proposed to reduce the discrepancy between source and target domain features [9, 10, 23, 26, 34, 38, 44, 50, 53, 60].”
Target-Independent Domain Adaptation for WBC Classification Using Generative Latent Search (2020, methodology): “Contrastive Adaptation Network (CAN) [35] estimates the underlying label hypothesis of target samples through clustering and adapts the feature representations according to the Contrastive Domain Discrepancy (CDD) metric.”
Learning to Learn Single Domain Generalization (2020, methodology): “To reduce the discrepancy across domains, a series of methods are proposed for unsupervised [33, 43, 7, 38, 39] or supervised domain adaptation [31, 57].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, methodology): “There are many other methods in the machine learning literature that aim to reduce data labeling efforts, such as transfer learning [149], domain adaptation [150]–[152], and semisupervised learning [153].”
Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Poorly-Drawn Sketches (2019, unknown): “In recent years, Generative Adversarial Networks (GANs) [23] have been successfully applied in many computer vision tasks to improve realism of generated images, such as domain adaption [24], [25], [26], super-resolution [27], [28], [29], [30], [31], [32].”
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations (2019, methodology): “In addition, I2I translation can be applied to synthesize images for domain adaptation [4,8,16,39,46].”
Unsupervised Attention-guided Image to Image Translation (2018, background): “Applications include image colorization [6], image super-resolution [7, 8], style transfer [9], domain adaptation [10] and data augmentation [11].”
Learning Semantic Segmentation From Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach (2018, background): “Moreover, inspired by the success of generative adversarial network [44, 17] and image translation techniques [60, 22], a few works have also suggested to transform synthetic images to realistic style, thus reducing domain gap on raw-pixel level [51, 20, 41, 20, 48, 16] and to boost the semantic seg”
Separate to Adapt: Open Set Domain Adaptation via Progressive Separation (2019, methodology): “Existing domain adaptation methods seek to bridge domain gap by distribution matching in feature-level [36, 4, 17, 19, 35, 3] or in pixellevel [10, 31, 11, 22, 16].”
DLOW: Domain Flow for Adaptation and Generalization (2018, background): “Recently, there is an increasing interest to apply domain adaptation techniques for semantic segmentation from synthetic data to the real scenario [22, 21, 7, 67, 40, 25, 11, 46, 51, 53, 23, 47, 62, 54, 43, 50, 52, 66, 5].”
Universal Domain Adaptation (2019, methodology): “Methods based on generative models synthesize labeled target samples as data augmentation and match domains in both pixel and feature levels [19, 36, 16, 20, 26, 17, 41].”
PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training (2021, background): “[38] trains a GANbased image-to-image translation network and a semantic segmentation network with the same backbone, and [32] extends the idea of cross-domain cycle-consistency to the multi-source setting.”
UM-Adapt: Unsupervised Multi-Task Adaptation Using Adversarial Cross-Task Distillation (2019, background): “This clearly makes CCR superior over the existing approaches of enforcing image reconstruction based regularization [1, 40, 50], by simplifying the additionally introduced decoder architecture devoid of the burden of generating irrelevant color-based appearance.”