Fisher GAN

By Youssef Mroueh, Tom Sercu. 2017

134 citations recorded by the source corpus.

Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN that fits within the Integral Probability Metrics (IPM) framework for training GANs. Fisher GAN defines a data dependent constraint on the second order moments of the critic. We show in this paper that Fisher GAN allows for stable and time efficient training that does not compromise the capacity of the critic, and does not need data independent constraints such as weight clipping. We analyze our Fisher IPM theoretically and provide an algorithm based on Augmented Lagrangian for Fisher GAN. We validate our claims on both image sample generation and semi-supervised classification using Fisher GAN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step (2017, methodology): “In Fisher-GAN [16], an equality constraint that is added on the magnitude of the output of the discriminator on data as well as samples is directly penalized, as opposed to the magnitude of the discriminator gradients, as in WGAN-GP.”
Demystifying MMD GANs (2018, background): “Many other GAN variants fall into the framework of IPMs (e.g. Mroueh et al., 2017; Mroueh & Sercu, 2017; Berthelot et al., 2017).”
Task2Vec: Task Embedding for Meta-Learning (2019, background): “FIM has also been proposed for various regularization schemes [5, 8, 22, 27], analyze learning dynamics of deep networks [4], and to overcome catastrophic forgetting [19].”
The relativistic discriminator: a key element missing from standard GAN (2018, neutral): “Many of the recent successful GANs variants have been based on Integral probability metrics (IPMs) [Müller, 1997] (e.g., WGAN [Arjovsky et al., 2017], WGAN-GP[Gulrajani et al., 2017], Sobolev GAN [Mroueh et al., 2017], Fisher GAN [Mroueh and Sercu, 2017]).”
Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks (2018, background): “(Radford et al., 2016) 6.54 ± 0.067 GAN - Improved GAN (Salimans et al., 2016) 6.86 ± 0.06 GAN + historical averaging - + minibatch discrimination EGAN-Ent-VI (Dai et al., 2017) 7.07 ± 0.10 Energy based Decoder net DFM (Warde-Farley & Bengio, 2017) 7.72 ± 0.13 Energy based Decoder net WGAN-GP (Gulra”
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models (2024, background): “Since then, IPMs have become crucial in GAN design (Mroueh & Sercu, 2017; Gulrajani et al., 2017), particularly in constraining the discriminator to a specific function class, thereby preventing it from overpowering the generator.”
Symmetric Variational Autoencoder and Connections to Adversarial Learning (2017, background): “Different classes of functions fψ(x) yield different learning metrics, including energy-based GAN [28], Maximum Mean Discrepancy [10], Fisher GAN [16], Wasserstein GAN (WGAN) [2], McGAN [17], among many other possible forms.”
How Generative Adversarial Networks and Its Variants Work : An Overview of GAN (2017, methodology): “4 Fisher GAN Instead of using standard IPM in equation 5, FisherGAN [19] focuses on standardized mean discrepancy which naturally induces a data dependent constraint as gradient penalty by following equation.”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “New Divergences: New families of Integral Probability Metrics (IPMs) for training GANs such as Fisher GAN [455], [456], mean and covariance feature matching GAN (McGan) [457], and Sobolev GAN [458], have been proposed.”
MINE: Mutual Information Neural Estimation (2018, methodology): “…estimation of f -divergences (Nguyen et al., 2010) and integral probability metrics (IPMs, Sriperumbudur et al., 2009) using deep neural networks (e.g., in the context of f -divergences and the Wasserstein distance or Fisher IPMs, Nowozin et al., 2016; Arjovsky et al., 2017; Mroueh & Sercu, 2017).”
Point Cloud GAN (2018, methodology): “On the other hand, in addition to weight clipping, several constraints (regularization) have bee proposed with better empirical performance, such as gradient penalty (Gulrajani et al., 2017) and L 2 ball (Mroueh and Sercu, 2017).”
Sphere Generative Adversarial Network Based on Geometric Moment Matching (2019, methodology): “In both discriminator networks, we used layer normalization [15] for the normalization unit suggested in [22], and we attached the geometric block GB to the last convolutional block for geometric transformation.”
Bridging the Gap Between $f$-GANs and Wasserstein GANs (2019, methodology): “Prior work have connected IPMs with χ2 divergences between mixtures of data and model distributions (Mao et al., 2017; Tao et al., 2018; Mroueh & Sercu, 2017); our approach can be applied to χ2 divergences as well, and we discuss its connections with χ2-GAN in Appendix C.”
Learning Implicit Generative Models by Matching Perceptual Features (2019, methodology): “For CIFAR10, STL10, LSUN and CelebA64×64, we use two extra layers as commonly used in previous works [28, 13].”
Calibration, Entropy Rates, and Memory in Language Models (2019, background): “…as a metric for the distance between two distributions sometimes goes under the name of integral probability metrics (Mller, 1997; Sriperumbudur et al., 2009), and this notion is becoming increasingly relevant again in unsupervised learning through the connections to GANs (Mroueh & Sercu, 2017).”
Stein Neural Sampler (2018, unknown): “, 2017), Chi-squared distance (Mroueh and Sercu, 2017), etc.”
Generative Adversarial Networks (GANs) (2020, background): “[55] also introduced a three-player game, called KDGAN, consists of a C, a teacher T, and a D in which C and T are trained from each other using distillation losses and are adversarially trained against D using adversarial losses.”
Learning Neural Random Fields with Inclusive Auxiliary Generators (2018, unknown): “13 Fisher-GAN [39] 7.”
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (2017, background): “Recent convergence proofs for GANs hold for expectations over training samples or for the number of examples going to infinity [32, 38, 35, 2], thus do not consider mini-batch learning which leads to a stochastic gradient [47, 23, 36, 33].”