Multi-agent Diverse Generative Adversarial Networks

By Arna Ghosh, Viveka Kulharia, Vinay P. Namboodiri, Philip H. S. Torr, Puneet Kumar Dokania. 2017

332 citations recorded by the source corpus.

We propose MAD-GAN, an intuitive generalization to the Generative Adversarial Networks (GANs) and its conditional variants to address the well known problem of mode collapse. First, MAD-GAN is a multi-agent GAN architecture incorporating multiple generators and one discriminator. Second, to enforce that different generators capture diverse high probability modes, the discriminator of MAD-GAN is designed such that along with finding the real and fake samples, it is also required to identify the generator that generated the given fake sample. Intuitively, to succeed in this task, the discriminator must learn to push different generators towards different identifiable modes. We perform extensive experiments on synthetic and real datasets and compare MAD-GAN with different variants of GAN. We show high quality diverse sample generations for challenging tasks such as image-to-image translation and face generation. In addition, we also show that MAD-GAN is able to disentangle different modalities when trained using highly challenging diverse-class dataset (e.g. dataset with images of forests, icebergs, and bedrooms). In the end, we show its efficacy on the unsupervised feature representation task.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Progressive Growing of GANs for Improved Quality, Stability, and Variation (2017, background): “That work in turn is motivated by Durugkar et al. (2016) who use one generator and multiple discrimi-nators concurrently, and Ghosh et al. (2017) who do the opposite with multiple generators and one discriminator.”
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs (2017, background): “Recently, several works learn to produce a fixed number of discrete outputs given the same input [5, 15] or synthesize diverse modes controlled by a latent code that encodes the entire image [64].”
Multimodal Unsupervised Image-to-Image Translation (2018, background): “To tackle this problem, some works propose to simultaneously generate multiple outputs given the same input and encourage them to be different [13, 45, 46].”
Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, methodology): “Reference [69] combined samples generated by different models to address mode collapse.”
Toward Multimodal Image-to-Image Translation (2017, background): “An effective option explored by concurrent work [2, 3, 13] is to use a mixture of models.”
A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020, background): “However, despite recent progress [13], generated images are still suboptimal for IL.”
Mutual Information Neural Estimation (2018, background): “Mode collapse (Che et al., 2016; Dumoulin et al., 2016; Donahue et al., 2016; Salimans et al., 2016; Metz et al., 2017; Saatchi & Wilson, 2017; Nguyen et al., 2017; Lin et al., 2017; Ghosh et al., 2017) is a common pathology of generative adversarial networks (GANs, Goodfellow et al.”
Video-to-Video Synthesis (2018, methodology): “As using multiple discriminators can mitigate the mode collapse problem during GANs training [19, 67,73], we also design two types of discriminators as detailed below.”
Mode matching in GANs through latent space learning and inversion (2018, background): “These works can be broadly categorized under the following two buckets - the ones that alter GAN’s structure and/or the learning objective for better mode discovery [1, 30, 29, 34, 13, 39, 7] and those which explicitly encourage GANs to discover diverse data modes through latent space inversion [36,”
On Data Augmentation for GAN Training (2020, methodology): “Instead of using only one discriminator or generator like standard GAN, the mixture models are proposed, such as multiple discriminators [39], [40], [41], the mixture of generators [42], [43] or an attacker applied as a new player for GAN training [44].”
Stochastic Conditional Generative Networks with Basis Decomposition (2019, methodology): “In [6], parameter non-uniqueness is used for multi-mode image generation by training several generators with different parameters simultaneously as a multi-agent solution.”
Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators (2018, background): “While some work has also focused on augmenting the number of generators [13, 14, 19], or even increasing both the number of generators and discriminators [7, 12, 16, 31], we will now turn our focus onmethods that solely increases the number of discriminators in order to prevent mode collapse.”
Every Smile is Unique: Landmark-Guided Diverse Smile Generation (2018, methodology): “Second, inspired by the multi-agent diverse GAN [11], we use the cross-entropy loss so as to discriminate between the sequences obtained from the K generators:”
MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks (2019, background): “More recently, a number of GAN-based methods break the process of high resolution image synthesis into smaller sub-tasks [36, 39, 38, 8, 9, 15, 34, 40].”
A Survey on ensemble learning under the era of deep learning (2021, background): “…mostly discussed about traditional ensemble learning, reviews (Cao et al. 2020) that particularly discussed ensemble deep learning in bioinformatics, or specific technical innovations with ensemble learning introduced such as GAN-Ensembles (Durugkar et al. 2017; Ghosh et al. 2018; Han et al. 2021).”
A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “using a variety of tricks [77, 155, 181, 192, 201], network architecture choices [101, 166, 181], objective modifications [4, 13, 54, 74, 97, 105, 138, 140, 142, 152– 154, 244], mixtures or ensembles [5, 51, 66, 80, 102, 147, 157, 207, 232], maximum mean discrepancy (MMD) [14, 52, 117, 120, 200], ma”
Evolutionary Generative Adversarial Networks (2018, methodology): “[49] trained many generators by using a multiclass discriminator that predicts which generator produces the sample.”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “Structure based methods: Representative methods in this category include such as MAD-GAN [46] and MRGAN [26] (detailed in Section 3.”
Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis (2019, methodology): “The other methods use auxiliary networks such as multiple generators [7, 18] and additional encoders [2, 4, 5, 27].”
Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation (2019, methodology): “Proposed solutions include layers which better condition the optimization, such as Spectral Normalization [60, 37], mod-iﬁcations to the loss function, such as WGAN [2, 18] or optimization procedure [21], or modeling proposals, such as MAD-GAN [16] and MUNIT [25].”