TinyGAN: Distilling BigGAN for Conditional Image Generation
By Ting-Yun Chang, Chi-Jen Lu. 2020
27 citations recorded by the source corpus.
Generative Adversarial Networks (GANs) have become a powerful approach for generative image modeling. However, GANs are notorious for their training instability, especially on large-scale, complex datasets. While the recent work of BigGAN has significantly improved the quality of image generation on ImageNet, it requires a huge model, making it hard to deploy on resource-constrained devices. To reduce the model size, we propose a black-box knowledge distillation framework for compressing GANs, which highlights a stable and efficient training process. Given BigGAN as the teacher network, we manage to train a much smaller student network to mimic its functionality, achieving competitive performance on Inception and FID scores with the generator having $16\times$ fewer parameters.
Subfields: Computer Vision and Pattern Recognition, Generative Models, Efficiency & Compression
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Axel Sauer
- Katja Schwarz
- Andreas Geiger
- Luc Van Gool
- Richard Zhang
- Ming-Hsuan Yang
- Eli Shechtman
- Andrea Vedaldi
In Generative Models
- Axel Sauer
- Katja Schwarz
- Andreas Geiger
- Tero Karras
- Timo Aila
- Richard Zhang
- Eli Shechtman
- Jun-Yan Zhu
In Efficiency & Compression
Citations in the corpus
- StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets (2022, background): “Therefore, it will be worth exploring GAN distillation methods [8] that trade-off performance for model size.”