Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data

By Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron C. Courville. International Conference on Machine Learning, 2018

465 citations recorded by the source corpus.

Learning inter-domain mappings from unpaired data can improve performance in structured prediction tasks, such as image segmentation, by reducing the need for paired data. CycleGAN was recently proposed for this problem, but critically assumes the underlying inter-domain mapping is approximately deterministic and one-to-one. This assumption renders the model ineffective for tasks requiring flexible, many-to-many mappings. We propose a new model, called Augmented CycleGAN, which learns many-to-many mappings between domains. We examine Augmented CycleGAN qualitatively and quantitatively on several image datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9e5be29ff2d6bf97607c31ab298590e75812143c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

StarGAN v2: Diverse Image Synthesis for Multiple Domains (2019, background): “nection between stochastic noise and the generated image for diversity, by marginal matching [1], latent regression [40, 13], and diversity regularization [35, 27].”
Emotional Voice Conversion: Theory, Databases and ESD (2021, methodology): “, 2017), Augmented ycleGAN (Almahairi et al., 2018) and StarGAN (Choi et al.”
Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic Segmentation (2019, background): “To produce multi-modal outputs, one possible approach injects noise vectors as additional inputs to the generator [54, 1, 11], but it could lead to the mode collapse problem.”
Few-Shot Unsupervised Image-to-Image Translation (2019, background): “There are works enforcing the cycle consistency constraint [51, 54, 25, 1, 55].”
Using latent space regression to analyze and leverage compositionality in GANs (2021, background): “In our work, we do not require clusters or concepts in intermediate layers to be defined a priori, nor do we need distinct input and output domains for approximate collages and real images, as in image translation tasks (Zhu et al., 2017; Almahairi et al., 2018).”
An Asymetric Cycle-Consistency Loss for Dealing with Many-to-One Mappings in Image Translation: A Study on Thigh MR Scans (2020, background): “rotation, shifting) could theoretically occur, it has been shown that this is unlikely [1,7].”
MirrorGAN: Learning Text-To-Image Generation by Redescription (2019, unknown): “Similar ideas to our own have recently been used in CycleGAN and DualGAN, which handle the bi-directional translations within two domains together [42, 36, 1], significantly advance image-to-image translation [14, 28, 15, 37, 23].”
On the Evaluation of Conditional GANs (2019, background): “Among the most promising approaches, Variational Auto-Encoders (VAEs) [20], auto-regressive models [43, 42] and Generative Adversarial Networks (GANs) [9] have been driving significant progress, with the latter at the forefront of a wide-range of applications [24, 52, 32, 46, 1, 38, 34].”
Unsupervised Domain-Specific Deblurring via Disentangled Representations (2019, background): “Recently, some unsupervised methods decouple images into domaininvariant content features and domain-specific attribute vectors, which produce diverse image-to-image translation outputs [17, 1, 11].”
Image-to-image translation for cross-domain disentanglement (2018, background): “Concurrently to our work, several approaches [1, 20, 27] have attempted to improve on image-toimage translation by disentangling the internal representation into content and style, which enable the effective generation of multi-modal outputs.”
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations (2019, background): “We note several concurrent methods (Almahairi et al. 2018; Cao et al. 2018; Huang et al. 2018; Lin et al. 2018a, b; Ma et al. 2018) (all independently developed) also adopt disentangled representations similar to our work for learning diverse I2I translation from unpaired training data.”
Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks (2019, methodology): “Novel modifications of the idea behind CycleGAN will likely solve this issue, for example the concept of augmented CycleGAN 12 .”
Unsupervised Object Segmentation by Redrawing (2019, methodology): “The technique has emerged from style modeling for style transfer tasks [12, 43] and has since been used for GANs as a mean to encode for style and to improve stochasticity [1, 8, 54].”
Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation (2019, methodology): “Similarly, employing FiLM modulation has been shown effective on a variety of tasks such as image synthesis [19, 36, 17, 1], visual question answering [38, 37], style transfer [7], recognition [16, 60], reading comprehension [5], etc.”
Visual Object Networks: Image Generation with Disentangled 3D Representations (2018, background): “We write the ﬁnal texture loss as Note that the latent space reconstruction loss L has been explored in unconditional GANs [Chen et al., 2016] and image-to-image translation [Zhu et al., 2017b, Almahairi et al., 2018].”
Exemplar Guided Unsupervised Image-to-Image Translation (2018, background): “However, we argue that these constraints are not well suited for complex domains with l rge inner-do ain variations, as also mentioned in [1, 14].”
Multimodal Unsupervised Image-to-Image Translation (2018, background): “A couple of concurrent works also recognize this limitation and propose extensions of CycleGAN/UNIT for multimodal mapping [47]/[48].”