Improved Consistency Regularization for GANs

By Zhengli Zhao, Sameer Singh, Honglak Lee, Zizhao Zhang, Augustus Odena. 2020

167 citations recorded by the source corpus.

Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator. We improve on this technique in several ways. We first show that consistency regularization can introduce artifacts into the GAN samples and explain how to fix this issue. We then propose several modifications to the consistency regularization procedure designed to improve its performance. We carry out extensive experiments quantifying the benefit of our improvements. For unconditional image synthesis on CIFAR-10 and CelebA, our modifications yield the best known FID scores on various GAN architectures. For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11.48 to 9.21. Finally, on ImageNet-2012, we apply our technique to the original BigGAN model and improve the FID from 6.66 to 5.38, which is the best score at that model size.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

Citations in the corpus

Training Generative Adversarial Networks with Limited Data (2020, methodology): “In contrast, a GAN trained under similar dataset augmentations learns to generate the augmented distribution [50, 53].”
A U-Net Based Discriminator for Generative Adversarial Networks (2020, background): “Note that BigGAN belongs to just one of the two state-of-the-art GAN families, led by BigGAN and StyleGAN, and their respective further improvements [51, 53, 21].”
Image Augmentations for GAN Training (2020, methodology): “Here we mainly focus on analyzing the efﬁcacy of different augmentations on BCR-GAN.”
Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs (2020, methodology): “…32, 44], learning meta-knowledge for quick adaptation to a target domain [24, 47, 42], using an auxiliary task to facilitate training [7, 26, 48, 49], improving an inference procedure of suboptimal models [2, 39, 29, 38], using an expressive prior distribution [13], actively choosing samples to…”
Improved Transformer for High-Resolution GANs (2021, methodology): “+ bCR [77] StyleGAN2 [28]† HiT-S HiT-B HiT-L”
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs (2021, background): “…adversarial loss, our work is related to recent advances in improving the sample quality and diversity of GANs, including data augmentation (Zhao et al., 2020; Karras et al., 2020a), consistency regularization (Zhang et al., 2020; Zhao et al., 2021) and entropy regularization (Dieng et al., 2019).”
ContraGAN: Contrastive Learning for Conditional Image Generation (2020, methodology): “Also, we think that exploring advanced regularization techniques [8, 9, 22, 23] is necessary to understand ContraGAN further.”
Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds (2023, background): “GANs are capable of rapid generation of high quality samples but suffer from unstable training and mode collapse in which parts of the data distribution are left uncaptured (Brock et al., 2019; Karras et al., 2020; Zhao et al., 2021).”
Training GANs with Stronger Augmentations via Contrastive Discriminator (2021, methodology): “We adopt a modiﬁed version of SNDCGAN architecture by Kurach et al. (2019) following the experimental setups of other baselines (Zhang et al., 2020; Zhao et al., 2020c).”
Data-Efficient Instance Generation from Instance Discrimination (2021, methodology): “Data Augmentation in GANs.”
Dual Contrastive Loss and Attention for GANs (2021, background): “Significant improvements are obtained by careful architectural designs for generators [10, 98, 42, 43, 69], discriminators [82, 56] and new regularization techniques [58, 3, 25, 60, 101, 105, 96, 40, 106, 36].”
One-Shot Generative Domain Adaptation (2021, background): “1) MineGAN works in two stages , where it first learns latent space transformation (with a frozen generator) and then fine-tunes the generator (with a frozen transformation).”
Regularizing Generative Adversarial Networks under Limited Data (2021, background): “To address these issues, recent meth-ods [30, 68, 80, 82, 83, 84] exploit data augmentation as a mean to increase data diversity, hence preventing the GAN models from overﬁtting the training data.”