Learning to Discover Cross-Domain Relations with Generative Adversarial Networks

By Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, Jiwon Kim. 2017

2,101 citations recorded by the source corpus.

While humans easily recognize relations between data from different domains without any supervision, learning to automatically discover them is in general very challenging and needs many ground-truth pairs that illustrate the relations. To avoid costly pairing, we address the task of discovering cross-domain relations when given unpaired data. We propose a method based on generative adversarial networks that learns to discover relations between different domains (DiscoGAN). Using the discovered relations, our proposed network successfully transfers style from one domain to another while preserving key attributes such as orientation and face identity.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Signal Processing

Citations in the corpus

Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis (2020, background): “, class information [3], [5], [6], [9], [36], source image [10], [12], [13], [37], text description [16], [17], [38], [39], scene graph [18], [40], etc) as input.”
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation (2017, methodology): “To alleviate this problem, we apply a cycle consistency loss [8, 32] to the generator, defined as”
Unsupervised Image-to-Image Translation Networks (2017, methodology): “arly, x 1= F 2!(x 2) = G (E(x 2)). The UNIT problem then becomes a problem of learning F1! and F2!1 . We note that a necessary condition for F1!2 and F2!1 to exist is the cycle-consistency constraint [29, 10]: x 1= F 2!(F (x 1)) and x2 = F!2(F (x2)). We can reconstruct the input image from translati”
Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization (2018, neutral): “This is conceptually related to several previous GAN models in the image domain that designed for joint distribution matching [13, 25, 35, 36, 37].”
cGANs with Projection Discriminator (2018, background): “…the discriminator and generator, and they have been drawing attention as a promising tool for class conditional image generation (Odena et al., 2017), the generation of the images from text (Reed et al., 2016; Zhang et al., 2017), and image to image translation (Kim et al., 2017; Zhu et al., 2017).”
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization (2017, methodology): “GANs have also been applied to style transfer [31] and cross-domain image generation [50, 3, 23, 38, 37, 25].”
Multimodal Unsupervised Image-to-Image Translation (2018, background): “Another popular constraint is the cycle consistency loss [7–9].”
Style Transfer from Non-Parallel Text by Cross-Alignment (2017, background): “Style transfer in vision Non-parallel style transfer has been extensively studied in computer vision (Gatys et al., 2016; Zhu et al., 2017; Liu and Tuzel, 2016; Liu et al., 2017; Taigman et al., 2016; Kim et al., 2017; Yi et al., 2017).”
Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, methodology): “Based on this situation, CycleGAN [49], DiscoGAN [50] and DualGAN [51] all adopt the idea of cyclic consistency, which can use unpaired data to train the mapping from X space to Y space.”
Synthesizing Tabular Data using Generative Adversarial Networks (2018, background): “GANs can generate high-quality images [36, 10, 22], and some models can even generate images conditioned on images or texts [28, 38, 20, 51, 23, 48].”
Deep Learning for Case-based Reasoning through Prototypes: A Neural Network that Explains its Predictions (2017, background): “Au-toencoders can be used for clustering (Xie, Girshick, and Farhadi 2015), generative modeling (Kingma and Welling 2014), discovering cross domain relations (Kim et al. 2017), and other tasks.”
Generative Feature Replay For Class-Incremental Learning (2020, background): “Generative adversarial networks (GANs) [11] are able to generate realistic and sharp images conditioned on object categories [12, 39], text [42, 57], another image (image translation) [18, 58] and style transfer [10].”
Moment Matching for Multi-Source Domain Adaptation (2018, background): “decoder [3, 10] or a GAN discriminator, such as dualGAN [46], cycle-GAN [50], disco-GAN [16], and CyCADA [15].”
Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations (2017, background): “Multiple works perform image-toimage translation between two unpaired images sets using adversarial training (Zhu et al. 2017; Kim et al. 2017; Yi et al. 2017; Fu et al. 2017; Taigman, Polyak, and Wolf 2017; Shrivastava et al. 2017; Bousmalis et al. 2017; Liu, Breuel, and Kautz 2017).”
CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017, methodology): “In order to encourage the source content to be preserved during the conversion process, we impose a cycle-consistency constraint on our adaptation method (Zhu et al., 2017; Yi et al., 2017; Kim et al., 2017) (see Figure 2 red portion).”
ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching (2017, methodology): “The cycle-consistency property has been previously leveraged in CycleGAN [12], DiscoGAN [13] and DualGAN [14].”
A Neural Representation of Sketch Drawings (2017, background): “Combining hybrid variations of sequence-generation models with unsupervised, cross-domain pixel image generation models, such as Image-to-Image models [3, 13, 18], is another exciting direction that we can explore.”
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, methodology): “CGANs can be trained even with unpaired training data using cycle-consistency loss (Kim et al. 2017; Yi et al. 2017; Zhu et al. 2017a).”