Learning Data Augmentation Strategies for Object Detection

By Barret Zoph, Ekin Dogus Cubuk, Golnaz Ghiasi, Tsung-Yi Lin, Jonathon Shlens, Quoc V. Le. 2019

607 citations recorded by the source corpus.

Data augmentation is a critical component of training deep learning models. Although data augmentation has been shown to significantly improve image classification, its potential has not been thoroughly investigated for object detection. Given the additional cost for annotating images for object detection, data augmentation may be of even greater importance for this computer vision task. In this work, we study the impact of data augmentation on object detection. We first demonstrate that data augmentation operations borrowed from image classification may be helpful for training detection models, but the improvement is limited. Thus, we investigate how learned, specialized data augmentation policies improve generalization performance for detection models. Importantly, these augmentation policies only affect training and leave a trained model unchanged during evaluation. Experiments on the COCO dataset indicate that an optimized data augmentation policy improves detection accuracy by more than +2.3 mAP, and allow a single inference model to achieve a state-of-the-art accuracy of 50.7 mAP. Importantly, the best policy found on COCO may be transferred unchanged to other detection datasets and models to improve predictive accuracy. For example, the best augmentation policy identified with COCO improves a strong baseline on PASCAL-VOC by +2.7 mAP. Our results also reveal that a learned augmentation policy is superior to state-of-the-art architecture regularization methods for object detection, even when considering strong baselines. Code for training with the learned policy is available online at this https URL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “Although this assumption is sufficient for identifying learned augmentation policies to improve performance [4, 51, 30, 19, 15], it is unclear if this assumption is overly stringent and may lead to sub-optimal data augmentation policies.”
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, neutral): “8 — — SpineNet-190 (1280) w/ self-training [72] 1885B 164M 54.”
EfficientDet: Scalable and Efficient Object Detection (2019, methodology): “32.4 2.5B YOLOv3[26] 33.0 71B(28x) EfﬁcientDet-D1 38.3 6.0B RetinaNet[17] 37.0 97B(16x) MaskRCNN[8] 37.9 149B(25x) EfﬁcientDet-D5 ‡ 49.8 136B AmoebaNet+NAS-FPN[37] ‡ 48.6 1317B(9.7x) EfﬁcientDet-D7 † 51.0 326B AmoebaNet+NAS-FPN[37] †‡ 50.7 3045B(9.3x) † Notplotted, ‡ trainedwithauto-augmentation[37]”
Visual Imitation Made Easy (2020, background): “More recently, learning augmentation strategies from data has emerged as a new paradigm to automate the design of augmentation [31, 32, 33].”
Adversarial Examples Improve Image Recognition (2019, background): “Recent works also demonstrate that it is possible to learn data augmentation policies automatically for achieving better performance on image classification [3, 4, 19, 21, 52] and object detection [53, 4].”
RODEO: Replay for Online Object Detection (2020, neutral): “Recently, several augmentation strategies have been designed explicitly for object detection [28, 54, 60] and it would be interesting to explore how they could improve performance within deep feature space for an incremental learning application.”
Revisiting ResNets: Improved Training and Scaling Strategies (2021, background): “Furthermore, [65] presents evidence that the addition of data augmentation shrinks the L2 norm of the weights, which renders some of the effects of weight decay redundant.”
Image Augmentations for GAN Training (2020, methodology): “In the image domain, a variety of augmentation techniques have been proposed to improve the performance of different visual recognition tasks such as image classiﬁcation [22, 13, 7], object detection [31, 47], and semantic segmentation [5, 15].”
Data Augmentation in Classification and Segmentation: A Survey and New Strategies (2023, background): “However, the increasing complexity of the model architectures with insufﬁcient data could exacerbate the shortcomings of CNN models [11].”
A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit (2021, methodology): “Color and geometric operations and changes inside the target object area are the main actions performed by data augmentation methods for object detection tasks [43].”
Thin MobileNet: An Enhanced MobileNet Architecture (2019, methodology): “Use of Random erasing in the network: Random erasing [19] is a kind of data augmentation [21, 27] method where we select rectangular regions in an image I Fig.”
Extracurricular Learning: Knowledge Transfer Beyond Empirical Distribution (2020, methodology): “These methods exploit domain knowledge to transform training examples to generate more data [51, 33, 17], learn a data generation policy [15, 14, 29, 35, 71, 62], augment the”
A Simple Semi-Supervised Learning Framework for Object Detection (2020, methodology): “Target boxes are augmented when global geometric transformations are used. detection [69], we design our augmentation strategy for object detection, which consists of global color transformation, global or box-level [69] geometric transformations, and Cutout [10].”
AntMan: Dynamic Scaling on GPU Clusters for Deep Learning (2020, methodology): “Training a DL model often requires a mixture of computations, some of which can hardly be parallelized using GPU, such as graph sampling in graph neural network [21, 54], feature extraction in advertisement [15, 23], data augmentation in computer vision [56], etc.”
Bearing Fault Detection and Diagnosis Using Case Western Reserve University Dataset With Deep Learning Approaches: A Review (2020, background): “are influential ML structures that work most excellent if trained on vast amounts of data [35].”
DAMO-YOLO : A Report on Real-Time Object Detection Design (2022, background): “However, recent advancement [3, 42] shows that properly designed box-level augmentation is crucial in object detection.”