DualGAN: Unsupervised Dual Learning for Image-to-Image Translation

By Zili Yi, Hao Zhang, Ping Tan, Minglun Gong. 2017

2,103 citations recorded by the source corpus.

Conditional Generative Adversarial Networks (GANs) for cross-domain image-to-image translation have made much progress recently [7, 8, 21, 12, 4, 18]. Depending on the task complexity, thousands to millions of labeled image pairs are needed to train a conditional GAN. However, human labeling is expensive, even impractical, and large quantities of data may not always be available. Inspired by dual learning from natural language translation [23], we develop a novel dual-GAN mechanism, which enables image translators to be trained from two sets of unlabeled images from two domains. In our architecture, the primal GAN learns to translate images from domain U to those in domain V, while the dual GAN learns to invert the task. The closed loop made by the primal and dual tasks allows images from either domain to be translated and then reconstructed. Hence a loss function that accounts for the reconstruction error of images can be used to train the translators. Experiments on multiple image translation tasks with unlabeled data show considerable performance gain of DualGAN over a single GAN. For some tasks, DualGAN can even achieve comparable or slightly better results than conditional GAN trained on fully labeled data.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, background): “The unpaired setting is also investigated when paired training data is not available (Choi et al., 2018; Huang et al., 2018; Kim et al., 2019; Lazarow et al., 2017; Liu et al., 2017; Yi et al., 2017; Zhao et al., 2020b; Zhu et al., 2017a).”
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs (2017, background): “Various methods have also been proposed to learn an image-to-image translation in the absence of training pairs [2, 33, 34, 46, 49, 51, 54, 63].”
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks (2017, background): “[55] independently introduce a similar objective for unpaired image-to-image translation, inspired by dual learning in machine translation [15].”
Achieving Human Parity on Automatic Chinese to English News Translation (2018, background): “Ever since dual learning was proposed, it has been successfully applied to various real-world problems such as question answering [35], image classification [47], image segmentation [25], image to image translation [50, 52, 24], face attribute manipulation [31], and machine translation [19, 43, 23, ”
Multimodal Unsupervised Image-to-Image Translation (2018, background): “Another popular constraint is the cycle consistency loss [7–9].”
Style Transfer from Non-Parallel Text by Cross-Alignment (2017, background): “Style transfer in vision Non-parallel style transfer has been extensively studied in computer vision (Gatys et al., 2016; Zhu et al., 2017; Liu and Tuzel, 2016; Liu et al., 2017; Taigman et al., 2016; Kim et al., 2017; Yi et al., 2017).”
Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, methodology): “Based on this situation, CycleGAN [49], DiscoGAN [50] and DualGAN [51] all adopt the idea of cyclic consistency, which can use unpaired data to train the mapping from X space to Y space.”
Synthesizing Tabular Data using Generative Adversarial Networks (2018, background): “GANs can generate high-quality images [36, 10, 22], and some models can even generate images conditioned on images or texts [28, 38, 20, 51, 23, 48].”
Moment Matching for Multi-Source Domain Adaptation (2018, background): “decoder [3, 10] or a GAN discriminator, such as dualGAN [46], cycle-GAN [50], disco-GAN [16], and CyCADA [15].”
Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations (2017, background): “Multiple works perform image-toimage translation between two unpaired images sets using adversarial training (Zhu et al. 2017; Kim et al. 2017; Yi et al. 2017; Fu et al. 2017; Taigman, Polyak, and Wolf 2017; Shrivastava et al. 2017; Bousmalis et al. 2017; Liu, Breuel, and Kautz 2017).”
CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017, methodology): “In order to encourage the source content to be preserved during the conversion process, we impose a cycle-consistency constraint on our adaptation method (Zhu et al., 2017; Yi et al., 2017; Kim et al., 2017) (see Figure 2 red portion).”
ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching (2017, methodology): “The cycle-consistency property has been previously leveraged in CycleGAN [9], DiscoGAN [10] and DualGAN [11].”
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, methodology): “CGANs can be trained even with unpaired training data using cycle-consistency loss (Kim et al. 2017; Yi et al. 2017; Zhu et al. 2017a).”
MirrorGAN: Learning Text-To-Image Generation by Redescription (2019, neutral): “Similar ideas to our own have recently been used in CycleGAN and DualGAN, which handle the bi-directional translations within two domains together [42, 36, 1], significantly advance image-to-image translation [14, 28, 15, 37, 23].”
Discriminator Rejection Sampling (2018, methodology): “They have also been applied successfully to semi-supervised and unsupervised learning (Springenberg, 2015; Odena, 2016; Kumar et al., 2017), image editing (Yu et al., 2018; Ledig et al., 2017), and image style transfer (Zhu et al., 2017; Isola et al., 2017; Yi et al., 2017; Azadi et al., 2018).”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, methodology): “Prior methods can be split into: (1) feature-level adaptation, where domain-invariant features are learned between source and target domains [17, 15, 57, 7, 14, 36, 5, 55], or (2) pixellevel adaptation, which focuses on re-stylizing images from the source domain to make them look like images from th”
Pros and Cons of GAN Evaluation Measures (2018, neutral): “Similar approaches have been followed in [40, 128, 120, 124, 129, 107, 18, 62, 66].”
Few-Shot Unsupervised Image-to-Image Translation (2019, background): “There are works enforcing the cycle consistency constraint [48, 51, 23, 1, 52].”