On Data Augmentation for GAN Training

By Ngoc-Trung Tran, Viet-Hung Tran, Ngoc-Bao Nguyen, Trung-Kien Nguyen, Ngai-Man Cheung. 2020

344 citations recorded by the source corpus.

Recent successes in Generative Adversarial Networks (GAN) have affirmed the importance of using more data in GAN training. Yet it is expensive to collect data in many domains such as medical applications. Data Augmentation (DA) has been applied in these applications. In this work, we first argue that the classical DA approach could mislead the generator to learn the distribution of the augmented data, which could be different from that of the original data. We then propose a principled framework, termed Data Augmentation Optimized for GAN (DAG), to enable the use of augmented data in GAN training to improve the learning of the original distribution. We provide theoretical analysis to show that using our proposed DAG aligns with the original GAN in minimizing the Jensen–Shannon (JS) divergence between the original distribution and model distribution. Importantly, the proposed DAG effectively leverages the augmented data to improve the learning of discriminator and generator. We conduct experiments to apply DAG to different GAN models: unconditional GAN, conditional GAN, self-supervised GAN and CycleGAN using datasets of natural images and medical images. The results show that DAG achieves consistent and considerable improvements across these models. Furthermore, when DAG is used in some GAN models, the system establishes state-of-the-art Fréchet Inception Distance (FID) scores. Our code is available (https://github.com/tntrung/dag-gans).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Training Generative Adversarial Networks with Limited Data (2020, background): “The third paper [43] presents a theoretical justification based on invertibility, but arrives at a different argument that leads to a more complex network architecture, along with significant restrictions on the set of possible augmentations.”
Projected GANs Converge Faster (2021, methodology): “We found that discriminator augmentation [25,63,72,73] consistently improves the performance of all methods, and is required to reach state-of-the-art performance.”
StyleGAN-NADA (2021, background): “Such works typically employ data augmentations [20, 50, 64, 65] or empower the discriminator to better learn from existing data using auxiliary tasks [29,60].”
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion (2021, background): “[14] in 2014, it has evolved considerably in terms of training strategies [2], [15], loss functions [16], [17], [18], regularizations [19], [20], [21], and network structures [22], [23].”
Diffusion-GAN: Training GANs with Diffusion (2022, background): “Tran et al. (2021) provide a theoretical analysis for deterministic non-leaking transformation with differentiable and invertible mapping functions.”
Data-Efficient Instance Generation from Instance Discrimination (2021, background): “Some recent attempts [24, 39, 49, 51, 44] have been made to apply data augmentation to GAN training.”
Dataset Condensation with Differentiable Siamese Augmentation (2021, background): “Differentiable Augmentation (Zhao et al., 2020a; Tran et al., 2020; Zhao et al., 2020b; Karras et al., 2020) has recently been applied for improving their training and in particular for preventing discriminators from memorizing the limited training set.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “If an augmentation is differentiable and represents an invertible transformation of the data space’s distribution, then the JS-divergence is invariant, and the generator is guaranteed to not create augmented samples [114], [220].”