Analyzing and Improving the Image Quality of StyleGAN
By Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila. 2019
7,107 citations recorded by the source corpus.
The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images. In addition to improving image quality, this path length regularizer yields the additional benefit that the generator becomes significantly easier to invert. This makes it possible to reliably attribute a generated image to a particular network. We furthermore visualize how well the generator utilizes its output resolution, and identify a capacity problem, motivating us to train larger models for additional quality improvements. Overall, our improved model redefines the state of the art in unconditional image modeling, both in terms of existing distribution quality metrics as well as perceived image quality.
Subfields: Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Andrea Vedaldi
- Andrew Zisserman
In Generative Models
- Jun-Yan Zhu
- Eli Shechtman
- Daniel Cohen-Or
- Richard Zhang
- Stefano Ermon
- Arash Vahdat
- Alexei A. Efros
- Karsten Kreis
Citations in the corpus
- High-Resolution Image Synthesis with Latent Diffusion Models (2021, background): “Generative Adversarial Networks (GAN) [26] allow for efficient sampling of high resolution images with good perceptual quality [3, 41], but are diffi-”
- Training Generative Adversarial Networks with Limited Data (2020, methodology): “Methods that do not employ augmentation (BigGAN, StyleGAN2, and our baseline) degrade noticeably as the size of the training set decreases, generally yielding poor image quality and diversity with fewer than 30k training images.”
- Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, background): “Despite the ability of GANs to generate high-resolution, sharp samples, the samples of GAN models sometimes contain bad artifacts or are even not recognizable [9].”
- CNN-Generated Images Are Surprisingly Easy to Spot… for Now (2019, methodology): “One such an evaluation scenario has naturally arisen, with the recent release of StyleGAN2 [23], a state-of-the-art unconditional GAN appearing in these proceedings.”
- Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors (2022, background): “Generative Adversarial Networks (GANs) [17] facilitate the generation of high fidelity images [29, 3, 30, 56] in multiple domains by simultaneously training a generator network G and a discriminator network D , where G is trained to fool D , while D is trained to judge if a given image is real or…”
- GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models (2021, background): “Meng et al. (2021) finds that diffusion models can not only inpaint regions of an image, but can do so conditioned on a rough sketch (or set of colors) for the image. Saharia et al. (2021a) finds that, when trained directly on the inpainting task, diffusion models can smoothly inpaint regions of an ”
- Generative Adversarial Networks (2021, neutral): “Style transfer using GANs has shown promising results in various domains, s.a. fashion design, art, and photography [19].”
- Towards Language-Free Training for Text-to-Image Generation (2021, background): “GANs [11] have inspired many advances in image synthesis [18, 20, 23, 28, 32].”
- Projected GANs Converge Faster (2021, methodology): “Following [26, 27], we report the FID between 50k generated and all real images.”
- Vector-quantized Image Modeling with Improved VQGAN (2021, methodology): “, 2018) based on VGG network (Simonyan & Zisserman, 2014) and GAN loss with architecture of StyleGAN discriminator (Karras et al., 2020).”
- D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, methodology): “However, generative adversarial networks (GANs, [34, 50]) and diffusion ∗Equal contribution.”
- On Fast Sampling of Diffusion Probabilistic Models (2021, neutral): “In other words, one has to run the forward-pass of the neural network T times to generate a sample, which is much slower than the state-of-the-art GANs or flow-based models for image and audio synthesis (e.g., Karras et al., 2020; Kingma and Dhariwal, 2018; Kong et al., 2020a; Ping et al., 2020).”
- Diffusion Models Beat GANs on Image Synthesis (2021, methodology): “This is for two reasons: first, some papers [33, 34, 31] compare against arbitrary subsets of the training set which are not readily available; and second, subtle implementation differences can affect the resulting FID values [51].”
- On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, methodology): “We train StyleGAN2 [38] (Config-E) models, using the identical training procedure and report FID of the result.”
- Noise Estimation for Generative Diffusion Models (2021, methodology): “(ii) non-autoregessive models, for example, StyleGAN [15] for vision application and WaveGAN [4] for audio synthesis.”
- Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, methodology): “We mostly borrow the network details and hyperparameters from StyleGAN2 (Karras et al., 2019b), including the number of convolutional layers (2) at each level, the number of channels (64 at 512×512 resolution, doubled at each coarser level with a maximum of 512), architecture of the mapping network…”
- Taming Transformers for High-Resolution Image Synthesis (2020, neutral): “0 StyleGAN2 (+aug) [30] 3.”
- This Face Does Not Exist... But It Might Be Yours! Identity Leakage in Generative Models (2020, methodology): “We used five different face matchers (face recognition, FaceNet, ArcFace, SphereFace and Neurotechnology MegaMatcher) and the StyleGAN2 synthesis model, and show that this identity leakage does exist for some, but not all methods.”