Feature Quantization Improves GAN Training
By Yang Zhao, Ping Yu, Jianfeng Gao, Changyou Chen. 2020
50 citations recorded by the source corpus.
The instability in GAN training has been a long-standing problem despite remarkable research efforts. We identify that instability issues stem from difficulties of performing feature matching with mini-batch statistics, due to a fragile balance between the fixed target distribution and the progressively generated distribution. In this work, we propose Feature Quantization (FQ) for the discriminator, to embed both true and fake data samples into a shared discrete space. The quantized values of FQ are constructed as an evolving dictionary, which is consistent with feature statistics of the recent distribution history. Hence, FQ implicitly enables robust feature matching in a compact space. Our method can be easily plugged into existing GAN models, with little computational overhead in training. We apply FQ to 3 representative GAN models on 9 benchmarks: BigGAN for image generation, StyleGAN for face synthesis, and U-GAT-IT for unsupervised image-to-image translation. Extensive experimental results show that the proposed FQ-GAN can improve the FID scores of baseline methods by a large margin on a variety of tasks, achieving new state-of-the-art performance.
Subfields: Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Generative Models
- Jinwoo Shin
- Jongheon Jeong
- Graham W. Taylor
- Changyou Chen
- Yang Zhao
- Terrance Devries
- Michal Drozdzal
- Yoshua Bengio
Citations in the corpus
- Latent Video Transformer (2020, unknown): “Discrete quantization was added to a discriminator in GAN [51].”
- Training GANs with Stronger Augmentations via Contrastive Discriminator (2021, methodology): “For this experiment, we exactly follow the training setups done in FQ-BigGAN (Zhao et al., 2020b Next, we evaluate the representation of the learned BigGAN encoder D from ContraD 5 under linear evaluation (on ImageNet) and several transfer learning benchmarks of natural image classification: namely,…”
- Instance Selection for GANs (2020, methodology): “Our best performing SAGAN model in terms of FID was trained on only 40% of the ImageNet dataset, yet outperforms FQ-BigGAN [34], the current state-of-the-art model for the task of 64 × 64 ImageNet generation.”
- Unpaired Image-to-Image Translation via Latent Energy Transport (2020, background): “Thus, these schemes generally require a combination of generators, discriminators or style encoders [25, 4, 5, 44], and complicate engineered loss designs.”