Large Scale GAN Training for High Fidelity Natural Image Synthesis
By Andrew Brock, Jeff Donahue, Karen Simonyan. 2018
6,195 citations recorded by the source corpus.
Despite recent progress in generative image modeling, successfully generating high-resolution, diverse samples from complex datasets such as ImageNet remains an elusive goal. To this end, we train Generative Adversarial Networks at the largest scale yet attempted, and study the instabilities specific to such scale. We find that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input. Our modifications lead to models which set the new state of the art in class-conditional image synthesis. When trained on ImageNet at 128x128 resolution, our models (BigGANs) achieve an Inception Score (IS) of 166.5 and Frechet Inception Distance (FID) of 7.4, improving over the previous best IS of 52.52 and FID of 18.6.
Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Eli Shechtman
- Daniel Cohen-Or
- Michael Niemeyer
- Jonathan T. Barron
- Alexei A. Efros
- Jun-Yan Zhu
- Richard Zhang
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
- Andrea Vedaldi
In Generative Models
- Jun-Yan Zhu
- Eli Shechtman
- Stefano Ermon
- Richard Zhang
- Arash Vahdat
- Daniel Cohen-Or
- Alexei A. Efros
- Jiaming Song
Citations in the corpus
- Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers (2020, background): “In the current deep learning paradigm, using more compute (e.g., increasing model size, dataset size, or training steps) typically leads to higher model accuracy (Brock et al., 2019; Raffel et al., 2019).”
- Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “Increasing Capacity Increasing the model capacity has been demonstrated to be a sure-shot approach to improving model quality in the presence of supervised data for several tasks, including Image Generation (Brock et al., 2018), Language Modeling and transfer learning (Radford et al.”
- ColdGANs: Taming Language GANs with Cautious Sampling Strategies (2020, background): “A natural alternative, not requiring human judgments, is to frame the problem under the Generative Adversarial Network (GAN) paradigm [14], which has been used successfully for image generation [3].”
- Exploring Self-Attention for Image Recognition (2020, neutral): “This has inspired applications of self-attention and related ideas to image recognition [5, 34, 15, 14, 45, 46, 13, 1, 27], image synthesis [43, 26, 2], image captioning [39, 41, 4], and video prediction [17, 35].”
- A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, methodology): “We also evaluate the encoder from B IG B I GAN (Donahue & Simonyan, 2019).”
- Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “Following recent successes showing that increasing network capacity and the scale of training result in improved performance [15, 7], we seek to grow our architecture while also maximizing the supervisory signal we obtain from each image.”
- Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, background): “Many of these approaches rely either on auto-encoding of images [24, 25, 26] or on adversarial learning [27], jointly modelling data and representation [28, 29, 30, 31].”
- Jukebox: A Generative Model for Music (2020, background): “, 2014) use generator and discriminator networks that contest each other to make the generated samples as indistinguishable as possible from the data, and they are renowned for their ability to generate high-quality pictures (Zhang et al., 2019b; Brock et al., 2019).”
- Evaluating Lossy Compression Rates of Deep Generative Models (2020, background): “Generative models of images represent one of the most exciting areas of rapid progress of AI (Brock et al., 2019; Kar-ras et al., 2018b;a).”
- Training Generative Adversarial Networks with Limited Data (2020, background): “This is in line with the common approximation (e.g., [4]) that FID roughly corresponds to Recall and IS to Precision, two independent aspects of result quality [24, 17].”
- Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, methodology): “Our unconditional model outperforms previous state-of-the-art GANs and other sampling-enhanced GANs [10–12] and even approaches the performance of conditional BigGANs [2] which achieves an Inception Score 9 .”
- CNN-Generated Images Are Surprisingly Easy to Spot… for Now (2019, methodology): “GANs We include three state-of-the-art unconditional GANs: ProGAN [21], StyleGAN [22], BigGAN [9], trained on either the LSUN [46] or ImageNet [40] datasets.”
- Analyzing and Improving the Image Quality of StyleGAN (2019, background): “The resolution and quality of images produced by generative methods, especially generative adversarial networks (GAN) [13], are improving rapidly [20, 26, 4].”
- Generative Modeling by Estimating Gradients of the Data Distribution (2019, background): “To list a few, they have been used to generate high-fidelity images [26, 6], synthesize realistic speech and music fragments [58], improve the performance of semi-supervised learning [28, 10], detect adversarial examples and other anomalous data [54], imitation learning [22], and explore promising s”
- Generating Diverse High-Fidelity Images with VQ-VAE-2 (2019, methodology): “For BigGan-deep the rejection sampling (referred to as critic) works better than the truncation method proposed in the BigGAN paper [5].”
- A Style-Based Generator Architecture for Generative Adversarial Networks (2018, methodology): “Here we used a variation of the truncation trick [38, 4, 31] with ψ = 0.”