Dreaming More Data: Class-dependent Distributions over Diffeomorphisms for Learned Data Augmentation

By Søren Hauberg, Oren Freifeld, Anders Boesen Lindbo Larsen, John W. Fisher III, Lars Kai Hansen. 2015

160 citations recorded by the source corpus.

Data augmentation is a key element in training high-dimensional models. In this approach, one synthesizes new observations by applying pre-specified transformations to the original training data; e.g.~new images are formed by rotating old ones. Current augmentation schemes, however, rely on manual specification of the applied transformations, making data augmentation an implicit form of feature engineering. With an eye towards true end-to-end learning, we suggest learning the applied transformations on a per-class basis. Particularly, we align image pairs within each class under the assumption that the spatial transformation between images belongs to a large class of diffeomorphisms. We then learn a class-specific probabilistic generative models of the transformations in a Riemannian submanifold of the Lie group of diffeomorphisms. We demonstrate significant performance improvements in training deep neural nets over manually-specified augmentation schemes. Our code and augmented datasets are available online.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (2015, methodology): “No data augmentation was applied to the images.”
Recent advances in convolutional neural networks (2015, methodology): “[114] propose an elegant way for data augmentation by randomly generating diffeomorphisms.”
Data Augmentation Generative Adversarial Networks (2017, background): “One paper that is worthy of note is the work of (Hauberg et al., 2016), where the authors learn augmentation strategies on a class by class basis.”
Learning to Compose Domain-Specific Transformations for Data Augmentation (2017, background): “[13] proposes learning a class-conditional model of diffeomorphisms interpolating between nearest-neighbor labeled data points as a way to perform augmentation.”
A Bayesian Data Augmentation Approach for Learning Deep Models (2017, background): “Hauberg et al. [13] proposed a manifold learning approach that is run once before the classiﬁer training begins, where this manifold describes the geometric transformations present in the training set.”
Data Augmentation in Emotion Classification Using Generative Adversarial Networks (2017, methodology): “Our approach aims to solve similar task in [11] but is very different from all these methods above.”
Adapting Auxiliary Losses Using Gradient Similarity (2018, methodology): “Different techniques have been studied for improving data efﬁciency, from data augmentation (Krizhevsky et al., 2012; Simonyan and Zisserman, 2015; Hauberg et al., 2016) to transfer learning (Taylor and Stone, 2009; Pan et al., 2010).”
Regularization for Deep Learning: A Taxonomy (2017, background): “Learning class-specific elastic image-deformation fields (Hauberg et al., 2016) p(θ|t,D) Input Random Training”
Augmenting Image Classifiers Using Data Augmentation Generative Adversarial Networks (2018, background): “One paper that is worthy of note is the work of [15], where the authors learn augmentation strategies on a class by class basis.”
Augmentor: An Image Augmentation Library for Machine Learning (2017, background): “Data augmentation is most commonly utilised in the branch of machine learning that concerns image analysis (Hauberg et al., 2016).”
Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology (2017, methodology): “The most common and straight forward way is to use data augmentation for constructing iterative optimization via the generation of artiﬁcial data or latent variables (Van Dyk and Meng, 2001; Hauberg et al., 2016).”
A Group-Theoretic Framework for Data Augmentation (2019, methodology): “Today, state-of-the-art algorithms can automatically learn appropriate transformations for data augmentation with generative models [54, 32, 4, 62], with reinforcement learning [57, 23], with Bayesian methods [65], or with some carefully designed search schemes [24].”
DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification (2018, background): “The authors of [25] learned class-conditional distributions by a diffeomorphism assumption.”
Further Advantages of Data Augmentation on Convolutional Neural Networks (2018, methodology): “New techniques have been proposed DeVries & Taylor (2017a,b) and, very interest-ingly, models that automatically learn useful data transformations have also been published lately Hauberg et al. (2016); Lemley et al. (2017); Ratner et al. (2017); Antoniou et al. (2017).”
Data augmentation instead of explicit regularization (2018, background): “Very interestingly, a promising avenue for future research has been set by recently proposed models that automatically learn the data transformations (Hauberg et al., 2016; Lemley et al., 2017; Ratner et al., 2017; Antoniou et al., 2017).”
Data Augmentation Using Learned Transformations for One-Shot Medical Image Segmentation (2019, methodology): “Parametric spatial transform models have been used to align and classify handwritten digits [31, 44, 50].”