High-Fidelity Image Generation With Fewer Labels

By Mario Lucic, Michael Tschannen, Marvin Ritter, Xiaohua Zhai, Olivier Bachem, Sylvain Gelly. 2019

175 citations recorded by the source corpus.

Deep generative models are becoming a cornerstone of modern machine learning. Recent work on conditional generative adversarial networks has shown that learning complex, high-dimensional distributions over natural images is within reach. While the latest models are able to generate high-fidelity, diverse natural images at high resolution, they rely on a vast quantity of labeled data. In this work we demonstrate how one can benefit from recent work on self- and semi-supervised learning to outperform the state of the art on both unsupervised ImageNet synthesis, as well as in the conditional setting. In particular, the proposed approach is able to match the sample quality (as measured by FID) of the current state-of-the-art conditional model BigGAN on ImageNet using only 10% of the labels and outperform it using 20% of the labels.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

Citations in the corpus

A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, methodology): “In the Cond-BigGAN and Uncond-BigGAN models we use publicly available 2 implementation by (Lucic et al., 2019) and (Chen et al., 2019) of the custom ResNet-like architecture proposed and described in (Brock et al., 2019).”
Diffusion Models Beat GANs on Image Synthesis (2021, methodology): “In the future, our method could be extended to unlabeled data by clustering samples to produce synthetic labels [30] or by training discriminative models to predict when samples are in the true data distribution or from the sampling distribution.”
Large Scale Adversarial Representation Learning (2019, methodology): “The lighter augmentation from [23] results in better image generation performance under the IS and FID metrics.”
Generative Feature Replay For Class-Incremental Learning (2020, background): “Only recently, the authors in [27] proposed to uses high resolution images.”
Continual learning with hypernetworks (2019, methodology): “, 2014) have become an established method for generative modelling and tend to produce higher quality images compared to VAEs, even at the scale of datasets as complex as ImageNet (Brock et al., 2019; Lučić et al., 2019; Donahue & Simonyan, 2019).”
Flow Matching for Generative Modeling (2022, neutral): “BigGAN (Luˇci´cetal.,2019) – 25.3 PGMGAN (Armandpouretal.,2021) – 21.7 FM w / OT 2.90 20.9 2017)), and averaged number of function evaluations (NFE) required for the adaptive solver to reach its a prespecified numerical tolerance, averaged over 50k samples.”
StarGAN v2: Diverse Image Synthesis for Multiple Domains (2019, background): “Generative adversarial networks (GANs) [9] have shown impressive results in many computer vision tasks such as image synthesis [3, 25, 7], colorization [15, 36] and superresolution [21, 34].”
Image Augmentations for GAN Training (2020, background): “For example, Self-Supervised GANs [6, 28] make the discriminator to predict the angle of rotated images and CRGAN [42] enforce consistency for different image perturbations.”
Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs (2020, methodology): “, transferring knowledge of a welltrained source model [45, 32, 44], learning meta-knowledge for quick adaptation to a target domain [24, 47, 42], using an auxiliary task to facilitate training [7, 26, 48, 49], improving an inference procedure of suboptimal models [2, 39, 29, 38], using an expressiv”
The Visual Task Adaptation Benchmark (2019, methodology): “We use both the label-conditional and unconditional BigGAN discriminators (Brock et al., 2019) (COND-BIGGAN and UNCOND-BIGGAN) using the public implementations of Chen et al. (2019); Lucic et al. (2019).”
Instance-Conditioned GAN (2021, unknown): “Using ﬁner partitions, however, tends to deteriorate results [33, 36, 42] because the clusters may contain too few data points for the generator and discriminator to properly model their data distribution.”
Partition-Guided GANs (2021, methodology): “Others [9, 49, 84] incorporated regularization on top of the augmentations, for example CRGAN [83] enforces consistency for different image perturbations.”
Semantic Bottleneck Scene Generation (2019, methodology): “For BigGAN, we followed the setup in [28] 1 , where we modiﬁed the code to allow for non-square images of Cityscapes.”
Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis (2019, background): “Recently, generative adversarial networks (GAN) [6] have shown stunning results in generating photorealistic images of faces [16, 17] and simple objects [34, 1, 22].”
A Large-Scale Study on Regularization and Normalization in GANs (2018, unknown): “These models were recently applied to great effect in a variety of applications, including image generation, learned compression, and domain adaptation (Brock et al., 2019; Menick & Kalchbrenner, 2019; Karras et al., 2019; Lucic et al., 2019; Isola et al., 2017; Tschannen et al., 2018).”
Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning (2024, background): “…BEGAN(Berthelot et al. 2017), CramerGAN(Bellemare et al. 2017), InfoMaxGAN(Lee et al. 2021), MMDGAN(Li et al. 2017), RelGAN(Nie et al. 2019), S3GAN(Luˇci´c et al. 2019), SNGAN(Miyato et al. 2018), and STGAN(Liu et al. 2019) The Thirty-Eighth AAAI Conference on Artiﬁcial Intelligence (AAAI-24)…”
Diverse Image Generation via Self-Conditioned GANs (2020, methodology): “Despite the remarkable progress of Generative Adversarial Networks (GANs) [13, 5, 20], there remains a significant gap regarding the quality and diversity between classconditionalGANs trained on labeled data, and unconditional GANs trained without any labels in a fully unsupervised setting [30, 26].”