Coupled Generative Adversarial Networks

By Ming-Yu Liu, Oncel Tuzel. 2016

1,720 citations recorded by the source corpus.

We propose coupled generative adversarial network (CoGAN) for learning a joint distribution of multi-domain images. In contrast to the existing approaches, which require tuples of corresponding images in different domains in the training set, CoGAN can learn a joint distribution without any tuple of corresponding images. It can learn a joint distribution with just samples drawn from the marginal distributions. This is achieved by enforcing a weight-sharing constraint that limits the network capacity and favors a joint distribution solution over a product of marginal distributions one. We apply CoGAN to several joint distribution learning tasks, including learning a joint distribution of color and depth images, and learning a joint distribution of face images with different attributes. For each task it successfully learns the joint distribution without any tuple of corresponding images. We also demonstrate its applications to domain adaptation and image transformation.

Subfields: Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs (2017, background): “Various methods have also been proposed to learn an image-to-image translation in the absence of training pairs [2, 33, 34, 46, 49, 51, 54, 63].”
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation (2017, methodology): “UNIT [16] combines variational autoencoders (VAEs) [11] with CoGAN [17], a GAN framework where two generators share weights to learn the joint distribution of images in cross domains.”
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks (2017, methodology): “From left to right: input, BiGAN/ALI [6, 7], CoGAN [28], CycleGAN (ours), pix2pix [20] trained on paired data, and ground truth.”
Learning from Simulated and Unsupervised Images through Adversarial Training (2016, methodology): “[21] uses coupled GANs to learn a joint distribution over images from multiple modalities without requiring tuples of corresponding images, achieving this by a weight-sharing constraint that favors the joint distribution solution.”
Self-Attention Generative Adversarial Networks (2018, background): “GANs have achieved great success in various image generation tasks, including image-to-image translation [9, 40, 29, 14], image super-resolution [12, 28] and text-to-image synthesis [24, 23, 37].”
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization (2017, methodology): “GANs have also been applied to style transfer [31] and cross-domain image generation [50, 3, 23, 38, 37, 25].”
Adversarial Discriminative Domain Adaptation (2017, methodology): “For example, [11, 12] share weights and learn a symmetric mapping of both source and target images to the shared feature space, while [13] decouple some layers thus learning a partially asymmetric mapping.”
Style Transfer from Non-Parallel Text by Cross-Alignment (2017, background): “For example, CoupledGANs (Liu and Tuzel, 2016) employ weight-sharing between networks to learn cross-domain representation, whereas CycleGAN (Zhu et al.”
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks (2017, background): “The translated images clearly match the rotation feature of the input images while preserving visual features of car and face domain, respectively.”
Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View (2019, background): “face frontal view generation [50], generating new human poses [51], photos to emojis [52], photograph editing [53]–[55], face aging [56], [57], photo blending [58], super resolution [59]–[61], photo inpainting [62]–[64], video prediction [65], and 3D object generation [66], [67]; among”
Few-Shot Adversarial Domain Adaptation (2017, methodology): “The first two tasks includeM→ U , U →M, and followed the experimental setting in [12, 45, 33, 59, 49], which involves randomly selecting 2000 images from MNIST and 1800 images from USPS.”
Moment Matching for Multi-Source Domain Adaptation (2018, background): “The second category is the adversarial-based approach [24, 40].”
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks (2016, neutral): “The work perhaps most similar to ours is that of Liu and Tuzel [30] who introduce an architecture of a pair of coupled GANs, one for the source and one for the target domain, whose generators share their high-layer weights and whose discriminators share their low-layer weights.”
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation (2017, background): “Recent work by Liu and Tuzel [10], which we refer to as coupled GAN or CoGAN, also trains two GANs together to solve image translation problems without paired training data.”
CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017, methodology): “CyCADA uniﬁes prior feature-level (Ganin & Lempitsky, 2015; Tzeng et al., 2017) and image-level (Liu & Tuzel, 2016b; Bousmalis et al., 2017b; Shrivastava et al., 2017) adversarial domain adaptation methods together with cycle-consistent image-to-image translation techniques (Zhu et al., 2017), as…”