Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis

By Bingchen Liu, Yizhe Zhu, Kunpeng Song, Ahmed M. Elgammal. 2021

316 citations recorded by the source corpus.

Training Generative Adversarial Networks (GAN) on high-fidelity images usually requires large-scale GPU-clusters and a vast number of training images. In this paper, we study the few-shot image synthesis task for GAN with minimum computing cost. We propose a light-weight GAN structure that gains superior quality on 1024*1024 resolution. Notably, the model converges from scratch with just a few hours of training on a single RTX-2080 GPU, and has a consistent performance, even with less than 100 training samples. Two technique designs constitute our work, a skip-layer channel-wise excitation module and a self-supervised discriminator trained as a feature-encoder. With thirteen datasets covering a wide variety of image domains (The datasets and code are available at: https://github.com/odegeasslbc/FastGAN-pytorch), we show our model's superior performance compared to the state-of-the-art StyleGAN2, when data and computing budget are limited.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Projected GANs Converge Faster (2021, methodology): “We do not have a license to re-distribute these datasets, but we provide the URLs to enable reproducibility, similar to [35].”
StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets (2022, background): “[52] evaluate several combinations of F andG and find an EfficientNet-Lite0 [56] and a FastGAN generator [37] to work especially well.”
Improved Techniques for Training Consistency Models (2023, methodology): “Here we exclude methods based on FastGAN (Liu et al., 2020; Sauer et al., 2021) or StyleGAN-XL (Sauer et al., 2022) from our consideration, because both utilize ImageNet pre-trained feature extractors in their discriminators.”
Musika! Fast Infinite Waveform Music Generation (2022, unknown): “We choose to adapt the FastGAN [33] architecture to our speciﬁc task.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “To learn long-range dependencies, GANs can be built with selfattention components [104], [234], [253], however, full quadratic attention does not scale well to high dimensional data.”
Diffusion-GAN: Training GANs with Diffusion (2022, methodology): “For ProjectedGAN models, we use the recommended default config (Sauer et al., 2021), which is based on FastGAN (Liu et al., 2020).”
LFS-GAN: Lifelong Few-Shot Image Generation (2023, background): “Other studies on the unconditional setting are FastGAN [30], which proposes a fast and stabilized GAN architecture and a self-supervised learning method, and MoCA [28], which employs a prototype memory with an attention mechanism [43].”
Fast Sampling of Diffusion Models with Exponential Integrator (2022, methodology): “Without any ad-hoc tricks, the plugin-and-play iPNDM is comparable or even slightly better than A-DDIM when the NFE budget is small, and DEIS is better than both of them.”