Further Advantages of Data Augmentation on Convolutional Neural Networks

By Alex Hernandez-Garcia, Peter König. 2018

126 citations recorded by the source corpus.

Data augmentation is a popular technique largely used to enhance the training of convolutional neural networks. Although many of its benefits are well known by deep learning researchers and practitioners, its implicit regularization effects, as compared to popular explicit regularization techniques, such as weight decay and dropout, remain largely unstudied. As a matter of fact, convolutional neural networks for image object classification are typically trained with both data augmentation and explicit regularization, assuming the benefits of all techniques are complementary. In this paper, we systematically analyze these techniques through ablation studies of different network architectures trained with different amounts of training data. Our results unveil a largely ignored advantage of data augmentation: networks trained with just data augmentation more easily adapt to different architectures and amount of training data, as opposed to weight decay and dropout, which require specific fine-tuning of their hyperparameters.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Albumentations: fast and flexible image augmentations (2018, background): “Image augmentations have also been shown to improve convergence [7], generalization and robustness on out-of-distribution samples [8,9], and to overall have more advantages compared to other regularization techniques [10].”
Data augmentation instead of explicit regularization (2018, methodology): “This section describes the experimental setup for systematically analyzing the role of data augmentation in deep neural networks compared to weight decay and dropout and builds upon the methods used in preliminary studies (Hernández-García and König, 2018a,b).”
Randomness In Neural Network Training: Characterizing The Impact of Tooling (2021, background): “For example, random initialization 88 [17, 21], data augmentation [38, 23], data shuffling ordering [73], and stochastic layers [76, 24, 81, 89 53, 49].”
A Group-Theoretic Framework for Data Augmentation (2019, background): “For example, a line of work proposes to add random or adversarial noise when training [11, 51, 26, 74, 8, 49, 73, 37] and argue that this can lead to a form of regularization [63, 34, 35, 18, 71].”
Tradeoffs in Data Augmentation: An Empirical Study (2021, methodology): “With this deﬁnition, it is widely acknowledged that commonly-used augmentations act as regular-izers (Hernández-García & König, 2018a;b; Zhang et al., 2016; Dao et al., 2019).”