Is synthetic data from generative models ready for image recognition?
By Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenqing Zhang, Philip H. S. Torr, Song Bai, Xiaojuan Qi. International Conference on Learning Representations, 2022
424 citations recorded by the source corpus.
Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images. Though the results are astonishing to human eyes, how applicable these generated images are for recognition tasks remains under-explored. In this work, we extensively study whether and how synthetic images generated from state-of-the-art text-to-image generation models can be used for image recognition tasks, and focus on two perspectives: synthetic data for improving classification models in data-scarce settings (i.e. zero-shot and few-shot), and synthetic data for large-scale model pre-training for transfer learning. We showcase the powerfulness and shortcomings of synthetic data from existing generative models, and propose strategies for better applying synthetic data for recognition tasks. Code: https://github.com/CVMI-Lab/SyntheticData.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/2a29e1bcbed17c588ffbae1fea2af3baaab924b8
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohammad Norouzi
- David J. Fleet
- Simon Kornblith
- Chitwan Saharia
- Quoc V. Le
- Richard Baraniuk
- Daniel LeJeune
- Shekoofeh Azizi
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- David J. Fleet
- Trevor Darrell
- Jitendra Malik
- Andrew Zisserman
- Mohammad Norouzi
Citations in the corpus
- Synthetic Data from Diffusion Models Improves ImageNet Classification (2023, methodology): “He et al. [21] show that synthetic data generated with GLIDE [40] improves zero-shot and few-shot image classification performance.”
- Effective Data Augmentation With Diffusion Models (2023, neutral): “Previous work utilizing large pretrained generative models to produce synthetic data (He et al., 2022) has left an important question unanswered: are we sure they are working for the right reason?”
- Self-Consuming Generative Models Go MAD (2023, background): “Moreover, AI-synthesized data is increasingly popular [5–10] because it is convenient [11, 12], anonymous [13–16], can augment real data [17, 18], and can match AI models’ ever-increasing sizes [19–21].”