Understanding Data Augmentation for Classification: When to Warp?

By Sebastien C. Wong, Adam Gatt, Victor Stamatescu, Mark D. McDonnell. 2016

974 citations recorded by the source corpus.

In this paper we investigate the benefit of augmenting data with synthetically created samples when training a machine learning classifier. Two approaches for creating additional training samples are data warping, which generates additional samples through transformations applied in the data-space, and synthetic over-sampling, which creates additional samples in feature-space. We experimentally evaluate the benefits of data augmentation for a convolutional backpropagation-trained neural network, a convolutional support vector machine and a convolutional extreme learning machine classifier, using the standard MNIST handwritten digit dataset. We found that while it is possible to perform generic augmentation in feature-space, if plausible transforms for the data are known then augmentation in data-space provides a greater benefit for improving performance and reducing overfitting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

Citations in the corpus

Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules (2019, background): “Some additional approaches to generate augmented data include image combining (Inoue, 2018; Okafor et al., 2018), elastic distortions (Wong et al., 2016), and generative adversarial networks (Antoniou et al., 2017).”
Rethinking Experience Replay: a Bag of Tricks for Continual Learning (2020, methodology): “Data augmentation is an effective strategy for improving the generalization capabilities of a Deep Network [31].”
The Effectiveness of Data Augmentation in Image Classification using Deep Learning (2017, methodology): “Data augmentation guided by expert knowledge [14], more generic image augmentation [18], and has shown effective in image classiﬁcation [16].”
A Review on Deep Learning Techniques Applied to Semantic Segmentation (2017, positive): “Data augmentation is a common technique that has been proven to benefit the training of machine learning models in general and deep architectures in particular; either speeding up convergence or acting as a regularizer, thus avoiding overfitting and increasing generalization capabilities [25].”
Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs (2018, methodology): “To test what role P (X) and P (Y |X) play in the smoothness of the learned filters, we followed the method of Zhang et al. (2016), and trained networks on a modified versions of the CIFAR-10 dataset in which we replace the labels with uniform random labels (which are consistent over training epochs)”
Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems (2021, methodology): “First, before training the model, Astraea performs data augmentation [18] to alleviate global imbalance.”
UNCERTAINTY MODELING AND INTERPRETABILITY IN CONVOLUTIONAL NEURAL NETWORKS FOR POLYP SEGMENTATION (2018, background): “Database Training Validation Test Resolution CVC-Colon [5] [1-76], [98-148], [221-273].”
Convolutional Neural Networks with Data Augmentation Against Jitter-Based Countermeasures - Profiling Attacks Without Pre-processing (2017, methodology): “To compensate some undesired behaviour of the CNNs, we propose as well to embed them with Data Augmentation (DA) techniques [28, 32], recommended in the machine learning domain for improving performances: the latter technique consists in artificially generating traces in order to increase the size o”
Practical Recommendations for Replay-based Continual Learning Methods (2022, methodology): “Finally, we test the role of data augmentation [32] in a continual learning scenario (Sec.”
Accurate brain tumor detection using deep convolutional neural network (2022, background): “4737 solving the overfitting issue, increasing the dataset size, and making the model more robust [42,49,50].”
A Survey on Data Augmentation for Text Classification (2021, positive): “With the help of adversarial training, these automatic adversarial example generators can be used as data augmentation methods, as done, for example, in [23], [7], [24], or [25], in order for models to be less susceptible to such easy alterations.”