Generative Adversarial Networks
By Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, Yoshua Bengio. 2021
30,470 citations recorded by the source corpus.
Generative Adversarial Networks (GANs) are a type of deep learning techniques that have shown remarkable success in generating realistic images, videos, and other types of data. This paper provides a comprehensive guide to GANs, covering their architecture, loss functions, training methods, applications, evaluation metrics, challenges, and future directions. We begin with an introduction to GANs and their historical development, followed by a review of the background and related work. We then provide a detailed overview of the GAN architecture, including the generator and discriminator networks, and discuss the key design choices and variations. Next, we review the loss functions utilized in GANs, including the original minimax objective, as well as more recent approaches s.a. Wasserstein distance and gradient penalty. We then delve into the training of GANs, discussing common techniques s.a. alternating optimization, minibatch discrimination, and spectral normalization. We also provide a survey of the various applications of GANs across domains. In addition, we review the evaluation metrics utilized to assess the diversity and quality of GAN-produced data. Furthermore, we discuss the challenges and open issues in GANs, including mode collapse, training instability, and ethical considerations. Finally, we provide a glimpse into the future directions of GAN research, including improving scalability, developing new architectures, incorporating domain knowledge, and exploring new applications. Overall, this paper serves as a comprehensive guide to GANs, providing both theoretical and practical insights for researchers and practitioners in the field.
Subfields: Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
- Yi Yang
In Generative Models
- Stefano Ermon
- Jun-Yan Zhu
- Arash Vahdat
- Eli Shechtman
- Richard Zhang
- Daniel Cohen-Or
- Jiaming Song
- Karsten Kreis
Citations in the corpus
- Zero-Shot Text-to-Image Generation (2021, background): “(2016b) later demonstrated that using a generative adversarial network (Goodfellow et al., 2014), rather than a recurrent variational auto-encoder, improved image fidelity.”
- Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, background): “MoCoGAN [48] proposed the motion and content decomposed generative adversarial networks for video generation.”
- Resolution-robust Large Mask Inpainting with Fourier Convolutions (2021, background): “One of the first inpainting works in deep learning era [34] used a convnet with an encoder-decoder architecture trained in an adversarial way [11].”
- ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis (2021, methodology): “To recover an image from x 1 , we utilize a CNN decoder G , such that the reverse model is specified as ) Here, f rec denotes the perceptual similarity metric [20, 29, 16, 77] (known as LPIPS) and D φ denotes a patch-based adversarial discriminator [22].”
- CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, background): “• BigGan Optimization , in which images are produced using a pre-trained BigGAN generator.”
- Cascaded Diffusion Models for High Fidelity Image Generation (2021, background): “Diffusion models [24] have recently been shown to synthesize high quality images and audio [3, 10, 14, 27], which have long been led by other classes of generative models such as autoregressive models, GANs, VAEs, and flows [2, 5, 7, 9, 12, 13, 18, 29, 30, 32].”
- CogView: Mastering Text-to-Image Generation via Transformers (2021, background): “Then Generative Adversarial Nets [19] (GANs) began to dominate this task.”
- On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, methodology): “With the proliferation of generative modeling techniques, such as Generative Adversarial Networks (GANs) [28], accurately discerning which methods are performing better has become a critical aspect of the field.”
- Image Super-Resolution via Iterative Refinement (2021, background): “Our 2-stage model improves on VQVAE-2 [71], is comparable to deep BigGANs [41] at truncation factor of 1.5 but underperforms them a truncation factor of 1.0.”
- Symbolic Music Generation with Diffusion Models (2021, background): “Variational autoencoders (VAEs) (Kingma & Welling, 2014), generative adversarial networks (GANs) (Goodfellow et al., 2014), and autoregressive models (ARs) (Oord et al.”
- Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, positive): “…network M (8-layer MLP), layer-wise noise injection, style mixing regularization (with a probability of 0.5 instead), non-saturating logistic loss (Goodfellow et al., 2014) with R1 regularization (Mescheder et al., 2018) of γ = 10, and the Adam optimizer (Kingma & Ba, 2014) with a learning rate…”
- Taming Transformers for High-Resolution Image Synthesis (2020, background): “learns a GAN [19] on representations of an autoencoder and [20] on low-resolution wavelet coefficients which are then decoded to images with a learned generator.”
- VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, neutral): “These models are trained by maximizing the data likelihood under the model, and unlike generative adversarial networks (GANs) (Goodfellow et al., 2014), their training is usually stable and they cover modes in data more faithfully by construction.”
- Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lenses (2020, methodology): “In the presence of infinite training data, computation time and network capacity for the generator and the discriminator, this process ensures that the p GAN distribution generated by G converges to that of the training data p data (Goodfellow et al., 2014).”
- Diffusion Autoencoders: Toward a Meaningful and Decodable Representation (2021, background): “Besides producing impressive image samples, GANs [15] have been shown to learn meaningful latent spaces [27] with extensive studies on multiple derived spaces [24, 57] and various knobs and controls for conditional human face generation [21, 37, 55].”
- StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (2021, background): “Generative Adversarial Networks (GANs) [12] have revolutionized image synthesis, with recent style-based generative models [18, 19, 16] boasting some of the most realistic synthetic imagery to date.”
- Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search (2021, methodology): “The state of the art in the field of image generation is represented by Generative Adversarial Networks (GANs) (Goodfellow et al., 2014).”