Diverse Image Generation via Self-Conditioned GANs

By Steven Liu, Tongzhou Wang, David Bau, Jun-Yan Zhu, Antonio Torralba. 2020

122 citations recorded by the source corpus.

We introduce a simple but effective unsupervised method for generating diverse images. We train a class-conditional GAN model without using manually annotated class labels. Instead, our model is conditional on labels automatically derived from clustering in the discriminator’s feature space. Our clustering step automatically discovers diverse modes, and explicitly requires the generator to cover them. Experiments on standard mode collapse benchmarks show that our method outperforms several competing methods when addressing mode collapse. Our method also performs well on large-scale datasets such as ImageNet and Places365, improving both diversity and standard metrics (e.g., Fréchet Inception Distance), compared to previous methods.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, background): “As such, developing on lower-resolution versions of the original dataset is a common practice [46, 78, 81], such as 1024→256 on FFHQ or 256→ 128 on ImageNet.”
On Aliased Resizing and Surprising Subtleties in GAN Evaluation (2022, background): “As such, developing on lower-resolution versions of the original dataset is a common practice [42, 72, 75], such as 1024→256 on FFHQ or 256 → 128 on ImageNet.”
TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data (2022, methodology): “Further, using self-conditioning for robust multi-modal feature extraction can help gain training stability and allow generalization [32].”
Fair Attribute Classification through Latent Space De-biasing (2020, background): “Over the past few years, a number of works [18, 24, 25, 27, 34] improved GANs to generate more realistic images with better stability.”
NExT-GPT: Any-to-Any Multimodal LLM (2023, methodology): “Moreover, to generate high-quality content, a multitude of strong-performing methods have been proposed, such as Transformer [82, 101, 17, 24], GANs [53, 7, 93, 110], VAEs [81, 67], Flow models [73, 6] and the current state-of-the-art diffusion models [31, 64, 57, 22, 68].”
The Vendi Score: A Diversity Evaluation Metric for Machine Learning (2022, methodology): “For the higher-resolution datasets, we use the 50,000 precomputed samples provided by Dhariwal and Nichol (2021) 9 for ADM and StyleGAN models.”
Generative Adversarial Networks (GANs) (2020, background): “Reengineered Network Architecture (S1) Conditional generation (S11) cGANs [28], FCGAN [47], IRGAN [47], GRANs [48], LAPGAN [5], SGAN [43], IcGAN [49], BiCoGAN [50], MatAN [51], Selfconditioned GANs [52], AC-GANs [53], TripleGAN [54], KDGAN [55], ControlGAN [56]”
Improved Transformer for High-Resolution GANs (2021, methodology): “We also note that [8, 36] leverage auxiliary regularization techniques and our method is complementary to them.”
Partition-Guided GANs (2021, methodology): “Furthermore, PGMGAN outperforms several other baseline models, even over supervised class-conditional GANs, on both CIFAR-10 and Stacked-MNIST, as shown in Table 1.”
Instance-Conditioned GAN (2021, unknown): “While these methods are promising and yield visually appealing samples, their quality is still far from those obtained with class-conditional GANs.”
The Platonic Representation Hypothesis (2024, background): “Our hypothesis could explain the phenomenon that conditional generation is easier than unconditional (Mirza & Osindero, 2014; Liu et al., 2020; Sauer et al., 2022), as the data we condition on may have the same platonic structure as the data we are generating.”