Randaugment: Practical automated data augmentation with a reduced search space

By Ekin Dogus Cubuk, Barret Zoph, Jonathon Shlens, Quoc V. Le. 2019

4,297 citations recorded by the source corpus.

Recent work on automated augmentation strategies has led to state-of-the-art results in image classification and object detection. An obstacle to a large-scale adoption of these methods is that they require a separate and expensive search phase. A common way to overcome the expense of the search phase was to use a smaller proxy task. However, it was not clear if the optimized hyperparameters found on the proxy task are also optimal for the actual task. In this work, we rethink the process of designing automated augmentation strategies. We find that while previous work required a search for both magnitude and probability of each operation independently, it is sufficient to only search for a single distortion magnitude that jointly controls all operations. We hence propose a simplified search space that vastly reduces the computational expense of automated augmentation, and permits the removal of a separate proxy task.Despite the simplifications, our method achieves equal or better performance over previous automated augmentation strategies on on CIFAR-10/100, SVHN, ImageNet and COCO datasets. EfficientNet-B7, we achieve 85.0% accuracy, a 1.0% increase over baseline augmentation, a 0.6% improvement over AutoAugment on the ImageNet dataset. With EfficientNet-B8, we achieve 85.4% accuracy on ImageNet, which matches a previous result that used 3.5B extra images. On object detection, the same method as classification leads to 1.0-1.3% improvement over baseline augmentation. Code will be made available online.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

Citations in the corpus

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, methodology): “[87] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le.”
VOLO: Vision Outlooker for Visual Recognition (2021, methodology): “For data augmentation methods, we use CutOut [76], RandAug [11], and the Token Labeling objective with MixToken [32].”
Towards Robust Vision Transformer (2021, methodology): “Same with the augmentations like MixUp [52], AugMix [18], Ran-dAugment [5], patch-wise augmentation also benefit the model robustness.”
Training Generative Adversarial Networks with Limited Data (2020, background): “Second, we aim for a maximally diverse set of augmentations, inspired by the success of RandAugment [7].”
Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, methodology): “, 2018) regularization, in addition to RandAugment (Cubuk et al., 2020) as used in Jaegle et al.”
Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, methodology): “Note that a similar idea of random shufﬂe strategy was proposed in [9], however, it is designed for image classiﬁcation and object detection and could be instead used to augment HR images.”
Perceiver: General Perception with Iterative Attention (2021, methodology): “We include ResNet-50 results from (Cubuk et al., 2020), as these numbers use RandAugment and hence better match our training protocol.”
Imitating Interactive Intelligence (2020, methodology): “RandAugment is applied to the images, and a ResNet processes frames, converting them into a spatial array of vector embeddings.”
A Survey of Data Augmentation Approaches for NLP (2021, background): “Whereas there are well-accepted frameworks for DA for CV (e.g. default augmentation libraries in PyTorch, RandAugment (Cubuk et al., 2019)), there are no such "generalized" DA techniques for NLP.”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “We vary the learning rate, decay schedule, loss function, and minimum crop size in the data augmentation, and optionally apply RandAugment (Cubuk et al., 2020), mixup (Zhang et al.”
Multiview Transformers for Video Recognition (2022, methodology): “We follow [3] and apply the same data augmentation and regularization schemes [13,29,65,83], which were used by [68] to train vision transformers more effectively.”