Pixel-Level Domain Transfer

By Donggeun Yoo, Namil Kim, Sunggyun Park, Anthony S. Paek, In-So Kweon. 2016

331 citations recorded by the source corpus.

We present an image-conditional image generation model. The model transfers an input domain to a target domain in semantic level, and generates the target image in pixel level. To generate realistic target images, we employ the real/fake-discriminator as in Generative Adversarial Nets, but also introduce a novel domain-discriminator to make the generated image relevant to the input image. We verify our model through a challenging task of generating a piece of clothing from an input image of a dressed person. We present a high quality clothing dataset containing the two domains, and succeed in demonstrating decent results.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Improved Techniques for Training GANs (2016, background): “Several recent papers focus on improving the stability of training and the resulting perceptual quality of GAN samples [2, 3, 5, 6].”
Learning from Simulated and Unsupervised Images through Adversarial Training (2016, background): “[44] tackle pixel-level semantic transfer learning with GANs.”
Adversarial Discriminative Domain Adaptation (2017, background): “Some argue that a single shared representation should be learned [6], while others claim that only part of the representation should be shared to enable the most effective adaptation [7, 9].”
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks (2016, background): “[47] introduce the task of generating images of clothes from images of models wearing them, by training on corresponding pairs of the clothes worn by models and on a hanger.”
CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017, background): “Yoo et al. (2016) learns a source to target encoder-decoder along with a generative adversarial objective on the reconstruction which is is applied for predicting the clothing people are wearing.”
Generative Multi-Adversarial Networks (2016, background): “…images (e.g., MNIST (LeCun et al. (1998)), CIFAR (Krizhevsky (2009))), however, GANs have proven useful in a variety of application domains including learning censored representations (Edwards & Storkey (2015)), imitating expert policies (Ho & Ermon (2016)), and domain transfer (Yoo et al. (2016)).”
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning (2018, background): “Other approaches have sought to map datapoints from one domain to another [48, 59, 5, 60].”
Scribbler: Controlling Deep Image Synthesis with Sketch and Color (2016, methodology): “Conditional GANs have also been used to transform images into different domains such as a product images [51] or different artistic styles [26].”
Semantic Image Synthesis via Adversarial Learning (2017, background): “On the other hand, there are many proposed approaches [17, 35, 11, 42, 18, 40, 1, 37, 19, 12] that condition GAN on images to synthesize images for various tasks.”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, background): “Pixel-level domain adaptation differs from image-to-image translation techniques [23, 10, 67], which deal with the easier task of learning such a re-stylization from matching pairs of examples from both domains.”
Learning Beyond Human Expertise with Generative Models for Dental Restorations (2018, unknown): “Prior works have conditioned GANs [9] to tackle various image-to-image generation problems [33,34,35,36].”
Person Transfer GAN to Bridge Domain Gap for Person Re-identification (2017, background): “Since GAN proposed by Goodfellow et al. [7], many variants of GAN [24, 25, 29, 35, 17, 33, 16, 5, 22, 34, 14, 41] have been proposed to tackle different tasks, e.g. , natu-ral style transfer, super-resolution, sketch-to-image generation, image-to-image translation, etc .”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “These models typically learn a real-to-real [257], [258], [260], [261] or synthetic-toreal [255], [256], [262] mapping to render the image style from the labeled source to the unlabeled target domain, thus offering synthetic training data with pseudo labels.”
An Introduction to Image Synthesis with Generative Adversarial Nets (2018, methodology): “[35].”
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach (2018, background): “Domain Adaptation ( DA ) approaches (e.g., [ 16 , 19 , 23 , 44 ]) aim to tackle this by transferring knowledge from a source domain (training data) to an unlabeled target domain (test data) to reduce the discrepancy between the source and target data distributions, typically by exploring…”
Adapting Convolutional Neural Networks for Geographical Domain Shift (2019, background): “A different adversarial approach would be to apply direct style transfer between source and target domains ( domain transfer ) [4, 13, 16, 27, 36,38].”
ACE: Adapting to Changing Environments for Semantic Segmentation (2019, unknown): “There is a growing interest in synthesizing images with Generative Adversarial Networks (GANs) [65, 38, 29], which is formulated as a minimax game between a generator and a discriminator [11].”