Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules

By Daniel E. Ho, Eric Liang, Ion Stoica, Pieter Abbeel, Xi Chen. 2019

443 citations recorded by the source corpus.

A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment are computationally infeasible to run for the ordinary user. In this paper, we introduce a new data augmentation algorithm, Population Based Augmentation (PBA), which generates nonstationary augmentation policy schedules instead of a fixed augmentation policy. We show that PBA can match the performance of AutoAugment on CIFAR-10, CIFAR-100, and SVHN, with three orders of magnitude less overall compute. On CIFAR-10 we achieve a mean test error of 1.46%, which is a slight improvement upon the current state-of-the-art. The code for PBA is open source and is available at this https URL.

Subfields: Computer Vision and Pattern Recognition, Economics and Econometrics, Education, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Economics and Econometrics

In Education

In Management Science and Operations Research

Citations in the corpus

Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “Although this assumption is sufﬁcient for identifying learned augmentation policies to improve performance [5, 57, 36, 25, 20], it is unclear if this assumption is overly stringent and may lead to sub-optimal data augmentation policies.”
VEGA: Towards an End-to-End Configurable AutoML Pipeline (2020, neutral): “PBA (Ho et al., 2019).”
FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function (2020, methodology): “These methods use policy search [4], population-based training [12], and Bayesian-based augmentation [35].”
UniformAugment: A Search-free Probabilistic Data Augmentation Approach (2020, methodology): “We compared UA results with those published by other recent methods, namely AA [3], FAA [12], PBA [8] and RA [4] 3 , as well as Baselines which only include the default augmentations for of the following models: WideResNet [17], Shake-Shake [6], ResNet-50 [7] and ResNet-200 [16].”
DADA: Differentiable Automatic Data Augmentation (2020, positive): “Specifically, our search space contains the same 15 data augmentation operations as PBA [10], which is also the same as AA and Fast AA except that the SamplePairing [11] is removed.”
Adversarial AutoAugment (2019, background): “Population based augmentation (PBA) (Ho et al., 2019) replaces the fixed augmentation policy with a dynamic schedule of augmentation policy along with the training process, which is mostly related to our work.”
Faster AutoAugment: Learning Augmentation Strategies using Backpropagation (2019, methodology): “One direction is to search for better combinations of symbolic operations using black-box optimization techniques such as reinforcement learning [5,24], evolution strategy [32], Bayesian optimization [18] and population-based training [12].”
Albumentations: fast and flexible image augmentations (2018, background): “Population-Based Augmentation [41] focused on generating augmentation policy schedules instead of fixed augmentation recipes.”
Visual Imitation Made Easy (2020, background): “More recently, learning augmentation strategies from data has emerged as a new paradigm to automate the design of augmentation [31, 32, 33].”
Learning Data Augmentation Strategies for Object Detection (2019, methodology): “More recently, several papers used the AutoAugment [5] search space with improved the optimization algorithms to ﬁnd AutoAugment policies more efﬁciently [17, 23].”
Optimized Generic Feature Learning for Few-shot Classification across Domains (2020, background): “The use of HPO for improving the test performance of deep networks is very common [4, 12, 20, 23, 30].”
Time Series Data Augmentation for Deep Learning: A Survey (2020, methodology): “For time series data augmentation, the MODALS [ Cheung and Yeung, 2021 ] is designed to ﬁnd the optimal composition of latent space transformations for data augmentation us-ing evolution search strategy based on population based augmentation (PBA) [ Ho et al. , 2019 ] , which demonstrates superior p”
Image Data Augmentation for Deep Learning: A Survey (2022, background): “Meanwhile, [Ho et al., 2019] proposes Population Based Augmentation (PBA) to reduce the time cost of AutoAugment which generates nonstationary augmentation policy schedules instead of a fixed augmentation policy.”
A Survey of Deep Learning for Scientific Discovery (2020, methodology): “…data augmentations, the related randaugment [38], choosing a random subset of diﬀerent possible augmentations, population based augmentation [85] which randomly searches over diﬀerent augmentation policies, [91] applying color distortions to the image and the recently proposed augmix [82]…”