AutoAugment: Learning Augmentation Policies from Data

By Ekin Dogus Cubuk, Barret Zoph, Dandelion Mané, Vijay Vasudevan, Quoc V. Le. 2018

1,986 citations recorded by the source corpus.

Data augmentation is an effective technique for improving the accuracy of modern image classifiers. However, current data augmentation implementations are manually designed. In this paper, we describe a simple procedure called AutoAugment to automatically search for improved data augmentation policies. In our implementation, we have designed a search space where a policy consists of many sub-policies, one of which is randomly chosen for each image in each mini-batch. A sub-policy consists of two operations, each operation being an image processing function such as translation, rotation, or shearing, and the probabilities and magnitudes with which the functions are applied. We use a search algorithm to find the best policy such that the neural network yields the highest validation accuracy on a target dataset. Our method achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, SVHN, and ImageNet (without additional data). On ImageNet, we attain a Top-1 accuracy of 83.5% which is 0.4% better than the previous record of 83.1%. On CIFAR-10, we achieve an error rate of 1.5%, which is 0.6% better than the previous state-of-the-art. Augmentation policies we find are transferable between datasets. The policy learned on ImageNet transfers well to achieve significant improvements on other datasets, such as Oxford Flowers, Caltech-101, Oxford-IIT Pets, FGVC Aircraft, and Stanford Cars.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

Citations in the corpus

Unsupervised Data Augmentation for Consistency Training (2019, methodology): “We ﬁnd that the augmentation policies found by AutoAugment work well in our semi-supervised learning setting although AutoAugment optimizes model’s performance in a supervised learning setting.”
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2019, methodology): “We also use swish activation (Ramachandran et al., 2018; Elfwing et al., 2018), ﬁxed Au-toAugment policy (Cubuk et al., 2019), and stochastic depth (Huang et al., 2016) with drop connect ratio 0.2.”
Fixing the train-test resolution discrepancy: FixEfficientNet (2020, background): “Over the years, training and testing pre-processing procedures have evolved, but so far they have been optimized separately [2].”
Self-Supervised Learning of Video-Induced Visual Invariances (2019, background): “Effect of AutoAugment (AA) Table 2 shows the effect of using AA [8] instead of the augmentation mechanism from [58].”
Fixing the train-test resolution discrepancy (2019, background): “Over the years, training and testing pre-processing procedures have evolved to improve the performance of CNNs, but so far they have been optimized separately [8].”
Attention Augmented Convolutional Networks (2019, background): “As such, it would be interesting to see if using Attention Augmentation as a primitive in automated architecture search procedures proves useful to find even better models than those previously found in image classification [55], object detection [12], image segmentation [6] and other domains[5, 1, ”
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization (2020, background): “The space of augmentations that help robustness includes various types of noise [22, 27, 21], highly unnatural image transformations [10, 38, 39], or compositions of simple image transformations such as Python Imaging Library operations [4, 15].”
On Empirical Comparisons of Optimizers for Deep Learning (2019, neutral): “The 77.1% we obtain with NA DAM , although not as good as the 77.6% obtained using learned data augmentation by Cubuk et al. (2018), is better than the best existing published results using any of the more standard pre-processing pipelines (76.5%, due to Goyal et al. (2017) using M OMENTUM ).”
Do ImageNet Classifiers Generalize to ImageNet? (2019, methodology): “autoaug_pyramid_net [8, 19] https://github.”
Early Convolutions Help Transformers See Better (2021, methodology): “We use AutoAugment [7], mixup [52] ( α = 0 .”
Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “Following [4], we employ the same linear scale for indicating the strength of each transformation.”
Training Generative Adversarial Networks with Limited Data (2020, background): “, leads to increasing invariance to these semantics-preserving distortions — a highly desirable quality in a classifier [17, 8, 9].”
Sharpness-Aware Minimization for Efficiently Improving Generalization (2020, methodology): “We have ensured that our imple-mentations’ generalization performance in the absence of SAM matches or exceeds that reported in prior work (Cubuk et al., 2018; Lim et al., 2019) All results use basic data augmentations (horizontal ﬂip, padding by four pixels, and random crop).”
MixMatch: A Holistic Approach to Semi-Supervised Learning (2019, background): “For example, in image classification, it is common to elastically deform or add noise to an input image, which can dramatically change the pixel content of an image without altering its label [7, 43, 10].”
A Survey of Data Augmentation Approaches for NLP (2021, methodology): “It has received active attention in recent machine learning (ML) research in the form of well-received, general-purpose techniques such as UDA (Xie et al., 2020) (3.1), which used backtranslation (Sennrich et al., 2016), Au-toAugment (Cubuk et al., 2018), and RandAugment (Cubuk et al., 2020), and M ”