BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations

By Daiqing Li, Huan Ling, Seung Wook Kim, Karsten Kreis, Adela Barriuso, Sanja Fidler, Antonio Torralba. 2022

140 citations recorded by the source corpus.

Annotating images with pixel-wise labels is a time consuming and costly process. Recently, DatasetGAN [78] showcased a promising alternative - to synthesize a large labeled dataset via a generative adversarial network (GAN) by exploiting a small set of manually labeled, GAN generated images. Here, we scale DatasetGAN to ImageNet scale of class diversity. We take image samples from the class-conditional generative model BigGAN [5] trained on ImageNet, and manually annotate only 5 images per class, for all 1k classes. By training an effective feature segmentation architecture on top of BigGAN, we turn Big GAN into a labeled dataset generator. We further show that VQGAN [18] can similarly serve as a dataset generator, leveraging the already annotated data. We create a new ImageNet benchmark by labeling an additional set of real images and evaluate segmentation performance in a variety of settings. Through an extensive ablation study, we show big gains in leveraging a large generated dataset to train different supervised and self-supervised backbone models on pixel-wise tasks. Furthermore, we demonstrate that using our synthesized datasets for pre-training leads to improvements over standard ImageNet pre-training on several downstream datasets, such as PASCAL-VOC, MS-COCO, Cityscapes and chest X-ray, as well as tasks (detection, segmentation). Our benchmark will be made public and maintain a leaderboard for this challenging task. Project Page: https://nv-tlabs.github.io/big-datasetgan/

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

InstructPix2Pix: Learning to Follow Image Editing Instructions (2022, background): “As generative models continue to improve, there is growing interest in their use as a source of cheap and plentiful training data for downstream tasks [33, 47, 50, 58, 65, 66].”
StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets (2022, methodology): “Exploring new editing methods and dataset generation [Chai et al. 2021; Li et al. 2022] using StyleGAN-XL are exciting future avenues.”
Synthetic Data from Diffusion Models Improves ImageNet Classification (2023, methodology): “Unlike methods that use model-based rendering, here we focus on the use of data-driven generative models of natural images, for which GANs have remained the predominant approach to date [6, 17, 36].”