Bayesian GAN

By Yunus Saatci, Andrew Gordon Wilson. 2017

151 citations recorded by the source corpus.

Generative adversarial networks (GANs) can implicitly learn rich distributions over images, audio, and data which are hard to model with an explicit likelihood. We present a practical Bayesian formulation for unsupervised and semi-supervised learning with GANs. Within this framework, we use stochastic gradient Hamiltonian Monte Carlo to marginalize the weights of the generator and discriminator networks. The resulting approach is straightforward and obtains good performance without any standard interventions such as feature matching or mini-batch discrimination. By exploring an expressive posterior over the parameters of the generator, the Bayesian GAN avoids mode-collapse, produces interpretable and diverse candidate samples, and provides state-of-the-art quantitative results for semi-supervised learning on benchmarks including SVHN, CelebA, and CIFAR-10, outperforming DCGAN, Wasserstein GANs, and DCGAN ensembles.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Mutual Information Neural Estimation (2018, background): “Mode collapse (Che et al., 2016; Dumoulin et al., 2016; Donahue et al., 2016; Salimans et al., 2016; Metz et al., 2017; Saatchi & Wilson, 2017; Nguyen et al., 2017; Lin et al., 2017; Ghosh et al., 2017) is a common pathology of generative adversarial networks (GANs, Goodfellow et al., 2014), where…”
Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection (2019, methodology): “[60] use SG-MCMC to sample the parameters of a generative adversarial network [25] to increase model expressiveness.”
A Bayesian Approach to Generative Adversarial Imitation Learning (2018, methodology): “GAIL reformulates the imitation learning problem as a density matching problem, and makes use of generative adversarial networks (GANs) [Goodfellow et al., 2014].”
Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning (2019, methodology): “( ii ) In practice, empirical successes have been reported by training DNNs in relatively short time (Li et al., 2016b; Chen et al., 2014; Gan et al., 2016; Neelakantan et al., 2016; Saatchi & Wilson, 2017).”
BoTorch: Programmable Bayesian Optimization in PyTorch (2019, background): “The basic idea is that a sample ζ i from P ( f ( x ) | D ) can be expressed as a suitable (differentiable) deterministic transformation ζ i = h D ( x , i ) of an auxiliary base sample i drawn from some base distribution independent of x .”
Deep convolutional Gaussian processes (2018, background): “Remarkably we do not experience rank degeneracy, possibly due to the multiple channel mappings and the convolution structure. ere are several avenues for improved e ciency and modelling capacity. e Stochastic Gradient Hamiltonian Monte Carlo approach (Ma et al., 2015) has proven e cient in deep GPs ”
Generative Adversarial Network Training is a Continual Learning Problem (2018, background): “…generators (Hoang et al., 2018; Ghosh et al., 2018) or multiple discriminators (Durugkar et al., 2017; Neyshabur et al., 2017), while Bayesian GAN (Saatchi & Wilson, 2017) considered distributions on the parameters of both networks, but these all do not consider time as the source of the ensemble.”
ProbGAN: Towards Probabilistic GAN with Theoretical Guarantees (2018, methodology): “We compare with three baselines: 1) GAN (or DCGAN 2 ) : naively trained multiple generators in the vanilla GAN framework; 2) MGAN : Mixture GAN (Hoang et al., 2018) which is the start-of-art method to train GAN with multiple generators; 3) BGAN : Bayesian GAN (Saatci & Wilson, 2017).”
Synthetic Data for Deep Learning (2019, methodology): “which provides representations of full posterior distributions over the parameters, was provided by Saatchi and Wilson [512].”
Generative Adversarial Networks (GANs) (2020, methodology): “[17] reviewed various GANs methods from the perspectives of algorithms, theory, and applications.”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “om the same distribution of the training examples. Therefore, GANs have neither a likelihood nor a well-deﬁned posterior. There are early attempts towards this research direction such as Bayesian GAN [459]. Although we can use GANs to generate data, how can we measure the uncertainty of the well-tra”
Learning Scalable Deep Kernels with Recurrent Structure (2016, methodology): “Indeed, SG-HMC has recently been used for eﬃcient inference over network parameters in the Bayesian GAN (Saatchi and Wilson, 2017).”