AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks

By Yonggan Fu, Wuyang Chen, Haotao Wang, Haoran Li, Yingyan Lin, Zhangyang Wang. 2020

92 citations recorded by the source corpus.

The compression of Generative Adversarial Networks (GANs) has lately drawn attention, due to the increasing demand for deploying GANs into mobile devices for numerous applications such as image translation, enhancement and editing. However, compared to the substantial efforts to compressing other deep models, the research on compressing GANs (usually the generators) remains at its infancy stage. Existing GAN compression algorithms are limited to handling specific GAN architectures and losses. Inspired by the recent success of AutoML in deep compression, we introduce AutoML to GAN compression and develop an AutoGAN-Distiller (AGD) framework. Starting with a specifically designed efficient search space, AGD performs an end-to-end discovery for new efficient generators, given the target computational resource constraints. The search is guided by the original GAN model via knowledge distillation, therefore fulfilling the compression. AGD is fully automatic, standalone (i.e., needing no trained discriminators), and generically applicable to various GAN models. We evaluate AGD in two representative GAN tasks: image translation and super resolution. Without bells and whistles, AGD yields remarkably lightweight yet more competitive compressed models, that largely outperform existing alternatives. Our codes and pretrained models are available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhangyang Wang
2. Yunhe Wang
3. Jiashi Feng
4. Yingyan Lin
5. Wuyang Chen
6. Yonggan Fu
7. Xinyu Gong
8. Wei Zhang

In Computer Vision and Pattern Recognition

1. Zhangyang Wang
2. Yunhe Wang
3. Jiashi Feng
4. Wuyang Chen
5. Yingyan Lin
6. Wei Zhang
7. Yonggan Fu
8. Xinyu Gong

In Generative Models

1. Zhangyang Wang
2. Yunhe Wang
3. Wuyang Chen
4. Xinyu Gong
5. Jiashi Feng
6. Tong Zhang
7. Shiyu Chang

In Neural Architecture Search & AutoML

1. Wuyang Chen
2. Zhangyang Wang
3. Yonggan Fu
4. Xinyu Gong
5. Yunhe Wang
6. Wei Zhang
7. Yingyan Lin
8. Tong Zhang

Citations in the corpus

• Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts (2020, background): “Neural Architecture Search [10, 11, 12, 13, 14] searches architectures that consume fewer resources with comparable performance.”
• Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search (2020, methodology): “Motivated by the promising performance achieved by those NAS works, recent works have extended NAS to more tasks such as segmentation [44, 10], image enhancement [20, 41], and language modeling [9].”
• Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective (2021, methodology): “Inspired by recent works on pruning-from-scratch (Lee et al., 2018; Wang et al., 2020), we propose a pruning-by-importance NAS mechanism to quickly shrink the search possibilities and boost the efficiency further, reducing the cost from |O| E to |O| · E .”