StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets

By Axel Sauer, Katja Schwarz, Andreas Geiger. 2022

702 citations recorded by the source corpus.

Computer graphics has experienced a recent surge of data-centric approaches for photorealistic and controllable content creation. StyleGAN in particular sets new standards for generative modeling regarding image quality and controllability. However, StyleGAN’s performance severely degrades on large unstructured datasets such as ImageNet. StyleGAN was designed for controllability; hence, prior works suspect its restrictive design to be unsuitable for diverse datasets. In contrast, we find the main limiting factor to be the current training strategy. Following the recently introduced Projected GAN paradigm, we leverage powerful neural network priors and a progressive growing strategy to successfully train the latest StyleGAN3 generator on ImageNet. Our final model, StyleGAN-XL, sets a new state-of-the-art on large-scale image synthesis and is the first to generate images at a resolution of 10242 at such a dataset scale. We demonstrate that this model can invert and edit images beyond the narrow domain of portraits or specific object classes. Code, models, and supplementary videos can be found at https://sites.google.com/view/stylegan-xl/ .

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Scalable Diffusion Models with Transformers (2022, methodology): “Our method achieves the lowest FID of all prior generative models, including the previous state-of-the-art StyleGANXL [50].”
Elucidating the Design Space of Diffusion-Based Generative Models (2022, methodology): “When using our training setup in CIFAR-10 (Figure 5b), the best results were obtained with deterministic sampling, and any amount of stochastic sampling was detrimental.”
The Stable Signature: Rooting Watermarks in Latent Diffusion Models (2023, background): “Image generation has long been dominated by GANs, still state-of-the-art on many datasets [37, 38, 39, 76, 87].”
Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models (2023, background): “The most powerful models today are built on generative adversarial networks [21, 38–40, 75 ], autoregressive transformers [ To this end, we build on latent diffusion models (LDMs), which can reduce the heavy computational burden when training on high-resolution images [65].”
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation (2024, methodology): “In Table 6, we compare with popular image generation models, including GAN [Brock et al. 2018; Kang et al. 2023; Sauer et al. 2022], Diffusion models [Dhariwal & Nichol 2021; Ho et al. 2022b; Rombach et al. 2022; Peebles & Xie 2023], and masked-prediction models [Chang et al. 2022].”
Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (2024, neutral): “First, there have been several approaches in generative adversarial network (GAN; Goodfellow et al. 2014) that try to accelerate training with better convergence using pretrained visual encoders (Sauer et al., 2021; Kumari et al., 2022; Sauer et al., 2022; 2023a; Kang et al., 2023).”
Improved Distribution Matching Distillation for Fast Image Synthesis (2024, methodology): “For instance, SDXL-Lightning [27] integrates a DiffusionGAN loss [25] with a progressive distillation objective [10, 13], while the Consistency Trajectory Model [26] combines a GAN [35] with an improved consistency distillation [9].”
Analyzing and Improving the Training Dynamics of Diffusion Models (2023, positive): “Model averaging [27, 49, 57, 68, 72] is an indispensable technique in all high-quality image synthesis methods [2, 11, 23, 29, 31, 33, 46, 48, 54, 60, 63, 65].”
StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis (2023, methodology): “Our architecture design is based on StyleGAN-XL (Sauer et al., 2022) that — similar to the original StyleGAN (Kar-ras et al., 2019) — ﬁrst processes the normally distributed input latent code z by a mapping network to produce an intermediate latent code w .”
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (2024, positive): “…transformers akin to GPT-2 and VQ-GAN [49, 19], with the only modification of substituting traditional layer normalization for adaptive normalization (AdaLN) – a choice motivated by its widespread adoption and proven effectiveness in visual generative models [33, 34, 32, 57, 56, 29, 46, 12].”
The Platonic Representation Hypothesis (2024, background): “Our hypothesis could explain the phenomenon that conditional generation is easier than unconditional (Mirza & Osindero, 2014; Liu et al., 2020; Sauer et al., 2022), as the data we condition on may have the same platonic structure as the data we are generating.”
Scaling up GANs for Text-to-Image Synthesis (2023, positive): “GAN2 [42] and 6× larger than StyleGAN-XL [86] and XMC-GAN [103].”
Unleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services (2023, neutral): “Image Generation StyleGAN [74], BigGANs [75], StyleGANXL [76], DVD-GAN [77], DALLE [6], DALLE2 [7],”
simple diffusion: End-to-end diffusion for high resolution images (2023, neutral): “Other types of generative models have also been successfully applied to image generation (Chang et al., 2022; Sauer et al., 2022; Anonymous, 2023), although modifications such as guidance and low temperature sampling can make it difficult to compare these models fairly.”
Improved Masked Image Generation with Token-Critic (2022, background): “Results for [3] are as reproduced by [34].”
Bridging CLIP and StyleGAN through Latent Alignment for Image Editing (2022, positive): “Besides, we have also integrated our method on Style-GAN3 [17] and StyleGAN-XL [34] for experiments on animal face dataset AFHQ [6, 7] and ImageNet [10].”