Image Augmentations for GAN Training

By Zhengli Zhao, Zizhao Zhang, Ting-Li Chen, Sameer Singh. 2020

149 citations recorded by the source corpus.

Data augmentations have been widely studied to improve the accuracy and robustness of classifiers. However, the potential of image augmentation in improving GAN models for image synthesis has not been thoroughly investigated in previous studies. In this work, we systematically study the effectiveness of various existing augmentation techniques for GAN training in a variety of settings. We provide insights and guidelines on how to augment images for both vanilla GANs and GANs with regularizations, improving the fidelity of the generated images substantially. Surprisingly, we find that vanilla GANs attain generation quality on par with recent state-of-the-art results if we use augmentations on both real and generated images. When this GAN training is combined with other augmentation-based regularization techniques, such as contrastive loss and consistency regularization, the augmentations further improve the quality of generated images. We provide new state-of-the-art results for conditional generation on CIFAR-10 with both consistency loss and contrastive loss as additional regularizations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Training Generative Adversarial Networks with Limited Data (2020, background): “While two of these papers [54, 51] propose essentially the same augmentation mechanism as we do, they study the absence of leak artifacts only empirically.”
Projected GANs Converge Faster (2021, methodology): “We found that discriminator augmentation [25,63,72,73] consistently improves the performance of all methods, and is required to reach state-of-the-art performance.”
Cross-Modal Contrastive Learning for Text-to-Image Generation (2021, background): “It has been explored under several adversarial training scenarios [25, 65, 9, 41].”
Differentiable Augmentation for Data-Efficient GAN Training (2020, methodology): “ta distribution itself. Directly applying augmentation would inevitably alter the distribution. We present a simple strategy to circumvent the above concern. Concurrent with our work, several methods [16,41,52] independently proposed data augmentation for training GANs. We urge the readers to check ”
Training GANs with Stronger Augmentations via Contrastive Discriminator (2021, methodology): “Self-supervised learning, including contrastive learning, have been recently applied in GAN as an auxiliary task upon the GAN loss (Chen et al., 2019; Tran et al., 2019; Lee et al., 2021; Zhao et al., 2020d), mainly in attempt to alleviate catastopic forgetting in discriminators (Chen et al., 2019).”
One-Shot Generative Domain Adaptation (2021, background): “Under such a case, however, we manage to reuse the prior knowledge to the most extent ( i.e. , only training one layer in the generator and the discriminator each), and confirm that such an efficient scheme indeed facilitates generative domain adaptation and substantially outperforms existing…”
Regularizing Generative Adversarial Networks under Limited Data (2021, background): “To address these issues, recent meth-ods [30, 68, 80, 82, 83, 84] exploit data augmentation as a mean to increase data diversity, hence preventing the GAN models from overﬁtting the training data.”
On Data Augmentation for GAN Training (2020, background): “Concurrent with our work, [54]–[56] independently propose data augmentation for training GANs very recently.”
Partition-Guided GANs (2021, background): “Others [9, 49, 84] incorporated regularization on top of the augmentations, for example CRGAN [83] enforces consistency for different image perturbations.”
Dataset Condensation with Differentiable Siamese Augmentation (2021, background): “Differentiable Augmentation (Zhao et al., 2020a; Tran et al., 2020; Zhao et al., 2020b; Karras et al., 2020) has recently been applied for improving their training and in particular for preventing discriminators from memorizing the limited training set.”
Multi-Concept Customization of Text-to-Image Diffusion (2022, methodology): “To adapt with just a few training images, efficient training techniques [28,33,61,68,83,84] are often useful.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “Several approaches independently proposed applying augmentations to all discriminator inputs, allowing more substantial augmentations to be used [114], [220], [260], [261]; the training procedure for a WGANwith augmentations is”
Dual Contrastive Loss and Attention for GANs (2021, background): “The progress is mainly driven by large-scale datasets [18, 57, 91, 38, 54, 42], architectural tuning [10, 98, 42, 43, 69], and loss designs [58, 3, 25, 60, 39, 101, 105, 96, 40, 106, 36].”
Improving Augmentation and Evaluation Schemes for Semantic Image Synthesis (2020, background): “Most recently, [75] provided some guidelines of the effectiveness of various existing augmentation techniques for GAN training.”
Dataset Distillation by Matching Training Trajectories (2022, methodology): “…perform N gradient descent updates on the student parameters with respect to the classiﬁcation loss of the synthetic data: where A is the differentiable augmentation technique [17, 40, 48, 49] used in previous work [45], and α is the (trainable) learning rate used to update the student network.”
Data-Efficient Instance Generation from Instance Discrimination (2021, background): “Some recent attempts [24, 39, 49, 51, 44] have been made to apply data augmentation to GAN training.”