Data Augmentation in Emotion Classification Using Generative Adversarial Networks

By Xinyue Zhu, Yifan Liu, Jiahong Li, Tao Wan, Zengchang Qin. 2017

121 citations recorded by the source corpus.

It is a difficult task to classify images with multiple class labels using only a small number of labeled examples, especially when the label (class) distribution is imbalanced. Emotion classification is such an example of imbalanced label distribution, because some classes of emotions like \emph{disgusted} are relatively rare comparing to other labels like {\it happy or sad}. In this paper, we propose a data augmentation method using generative adversarial networks (GAN). It can complement and complete the data manifold and find better margins between neighboring classes. Specifically, we design a framework with a CNN model as the classifier and a cycle-consistent adversarial networks (CycleGAN) as the generator. In order to avoid gradient vanishing problem, we employ the least-squared loss as adversarial loss. We also propose several evaluation methods on three benchmark datasets to validate GAN's performance. Empirical results show that we can obtain 5%~10% increase in the classification accuracy after employing the GAN-based data augmentation techniques.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “GANs have also been used to generate training data directly [37, 33, 56, 1, 44], however this approach does not seem to be as beneﬁcial as learning sequences of data augmentation operations that are pre-deﬁned [40].”
Automatically Learning Data Augmentation Policies for Dialogue Tasks (2019, background): “…have also adopted Generative Adversarial Networks (Goodfellow et al., 2014) to either directly generate augmented data (Tran et al., 2017; Sixt et al., 2018; Antoniou et al., 2017; Zhu et al., 2017; Mun et al., 2017; Wang and Perez, 2017), or generate augmentation strategies (Ratner et al., 2017).”
GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks (2018, methodology): “In [1], the authors train a conditional GAN on unlabelled data to generate alternative versions of a given real image, and in [23], the authors use a similar GAN to impose emotions on neutral faces to expand underrepresented classes.”
Learning Data Augmentation Strategies for Object Detection (2019, methodology): “For image classification models, data can be augmented either by learning a generator that can create data from scratch [33, 31, 54, 2, 43], or by learning a set of transformations as applied to already existing training set samples [5, 37].”
AutoAugment: Learning Augmentation Strategies From Data (2019, methodology): “Generative adversarial networks have also been used for the purpose of generating additional data (e.g., [45, 41, 70, 2, 56]).”
Multi-hop Federated Private Data Augmentation with Sample Compression (2019, background): “Alternatively, generating synthetic data samples via generative adversarial networks (GANs) is a compelling data augmentation solution that guarantees data privacy while avoiding data exchanges [Sixt et al., 2018; Zhu et al., 2017].”
A survey on face data augmentation for the training of deep neural networks (2019, methodology): “The authors in [167] used this framework (whose architecture is shown in Fig.”
Ensembling with Deep Generative Views (2021, background): “Several works investigate using generative samples as data augmentation in an image translation setting, in which an autoencoder-style network is trained to produce variations of an input image, on a variety of domains such as face attribute editing [57], gaze estimation [54], emotion classification”
RandAugment: Practical data augmentation with no separate search (2019, methodology): “GANs have also been used to generate training data directly [36, 32, 55, 1, 43], however this approach does not seem to be as beneficial as learning sequences of data augmentation operations that are pre-defined [39].”
LARGE: Latent-Based Regression through GAN Semantics (2021, background): “Others enriched a real dataset to improve performance in the low-data domain [4,14,66,80] or on biased or unbalanced data [16,20,32,43,50].”
Fair Attribute Classification through Latent Space De-biasing (2020, methodology): “Using GANs to augment datasets: Several works use GANs to augment datasets for low-shot [20] and longtail [48] recognition tasks, whereas our work focuses specifically on de-biasing classifiers affected by dataset bias.”
Improving Augmentation and Evaluation Schemes for Semantic Image Synthesis (2020, methodology): “While in principle augmentation strategies can be applied to any class of models, their utility in improving the training of generative adversarial networks (GANs) for image synthesis has not been widely studied, and very little attention has been paid to developing novel augmentation strategies spe”