Improving GAN Training via Binarized Representation Entropy (BRE) Regularization

By Yanshuai Cao, Gavin Weiguang Ding, Kry Yik-Chau Lui, Ruitong Huang. 2018

19 citations recorded by the source corpus.

We propose a novel regularizer to improve the training of Generative Adversarial Networks (GANs). The motivation is that when the discriminator D spreads out its model capacity in the right way, the learning signals given to the generator G are more informative and diverse. These in turn help G to explore better and discover the real data manifold while avoiding large unstable jumps due to the erroneous extrapolation made by D. Our regularizer guides the rectifier discriminator D to better allocate its model capacity, by encouraging the binary activation patterns on selected internal layers of D to have a high joint entropy. Experimental results on both synthetic data and real datasets demonstrate improvements in stability and convergence speed of the GAN training, as well as higher sample quality. The approach also leads to higher classification accuracies in semi-supervised learning.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. David M. Blei
2. Adji Bousso Dieng
3. Michalis K. Titsias
4. Luc Van Gool
5. Francisco J. R. Ruiz
6. Trevor Darrell
7. Dacheng Tao
8. Jitendra Malik

In Statistical and Nonlinear Physics

1. Francisco J. R. Ruiz
2. Michalis K. Titsias
3. Roger Baker Grosse
4. David M. Blei
5. Huan Ling
6. Alireza Makhzani
7. Avishek Joey Bose
8. Adji Bousso Dieng

In Generative Models

1. Adji Bousso Dieng
2. Michalis K. Titsias
3. David M. Blei
4. Alireza Makhzani
5. Francisco J. R. Ruiz
6. Roger Baker Grosse
7. Huan Ling
8. Avishek Joey Bose

Citations in the corpus

• Evaluating Lossy Compression Rates of Deep Generative Models (2020, methodology): “, 2018), and BRE-GAN (Cao et al., 2018), with the code size of d ∈ {2, 10, 100}.”
• Prescribed Generative Adversarial Networks (2019, background): “Finally, Cao et al. (2018) introduce a regularizer for the discriminator to encourage diverse activation patterns in the discriminator across different samples.”
• PassGAN: A Deep Learning Approach for Password Guessing (2017, unknown): “[25], there have been several improvements on GANs [3, 4, 6–8, 10, 14, 16, 27, 35, 36, 42, 47, 53, 55, 56, 58–60, 63, 66, 69, 74, 76, 80, 87, 90, 94, 96], where each new paper provides novel improvements in the domain.”
• Adversarial Contrastive Estimation (2018, background): “Much work has been done to stabilize GAN training in the continuous case (Arjovsky et al., 2017; Gulrajani et al., 2017; Cao et al., 2018).”