RenderGAN: Generating Realistic Labeled Data

By Leon Sixt, Benjamin Wild, Tim Landgraf. 2016

180 citations recorded by the source corpus.

Deep Convolutional Neuronal Networks (DCNNs) are showing remarkable performance on many computer vision tasks. Due to their large parameter space, they require many labeled samples when trained in a supervised setting. The costs of annotating data manually can render the use of DCNNs infeasible. We present a novel framework called RenderGAN that can generate large amounts of realistic, labeled images by combining a 3D model and the Generative Adversarial Network framework. In our approach, image augmentations (e.g., lighting, background, and detail) are learned from unlabeled data such that the generated images are strikingly realistic while preserving the labels known from the 3D model. We apply the RenderGAN framework to generate images of barcode-like markers that are attached to honeybees. Training a DCNN on data generated by the RenderGAN yields considerably better performance than training it on various baselines.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “GANs have also been used to generate training data directly [37, 33, 56, 1, 44], however this approach does not seem to be as beneﬁcial as learning sequences of data augmentation operations that are pre-deﬁned [40].”
AutoAugment: Learning Augmentation Policies from Data (2018, methodology): “Generative adversarial networks have also been used for the purpose of generating additional data (e.g., [45–49]).”
Faster AutoAugment: Learning Augmentation Strategies using Backpropagation (2019, methodology): “[27, 28] use GANs to modify the outputs of simulators to look like real objects.”
Automatically Learning Data Augmentation Policies for Dialogue Tasks (2019, methodology): “…works have also adopted Generative Adversarial Networks (Goodfellow et al., 2014) to either directly generate augmented data (Tran et al., 2017; Sixt et al., 2018; Antoniou et al., 2017; Zhu et al., 2017; Mun et al., 2017; Wang and Perez, 2017), or generate augmentation strategies (Ratner et…”
Learning to Compose Domain-Specific Transformations for Data Augmentation (2017, background): “These procedures make the strong assumption that all of the provided transformations will preserve class labels, or use bespoke models over restricted sets of operations [28].”
Learning Data Augmentation Strategies for Object Detection (2019, methodology): “For image classification models, data can be augmented either by learning a generator that can create data from scratch [33, 31, 54, 2, 43], or by learning a set of transformations as applied to already existing training set samples [5, 37].”
Disentangling Adversarial Robustness and Generalization (2018, background): “, [74, 91, 4, 20], where training examples are generated, but without the goal to be misclassified.”
RandAugment: Practical data augmentation with no separate search (2019, methodology): “GANs have also been used to generate training data directly [36, 32, 55, 1, 43], however this approach does not seem to be as beneficial as learning sequences of data augmentation operations that are pre-defined [39].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “[309], [310], [311], [312], [313], [314], [315], [316], to generate neural networks [317], to augment data [318], [319], to learn spatial representations [320], and in network embedding [321], heterogeneous information networks [322], and”
Multi-hop Federated Private Data Augmentation with Sample Compression (2019, background): “Alternatively, generating synthetic data samples via generative adversarial networks (GANs) is a compelling data augmentation solution that guarantees data privacy while avoiding data exchanges [Sixt et al., 2018; Zhu et al., 2017].”
Training Discriminative Models to Evaluate Generative Ones (2018, background): “They can also be used for data augmentation [20], for safety against adversarial example [27], or to produce labeled data [24] in order to improve the training of discriminative models.”
Augment your batch: better training with larger batches (2019, background): “For example, Bayesian approaches based on the training set distribution (Tran et al., 2017), generative approaches based on generative adversarial networks (Antoniou et al., 2017; Sixt et al., 2018) and search methods aimed to ﬁnd the best data augmentation policy (Cubuk et al., 2018).”
Open Set Learning with Counterfactual Images (2018, background): “The use of generative adversarial networks for data augmentation has been explored in the context of image classification [23].”
Bidirectional Conditional Generative Adversarial Networks (2017, background): “This provides an information-rich representation of data for auxiliary supervised semantic tasks [4], as well as a way for conditional data augmentation [27,28] to aid their learning.”
AutoFlow: Learning a Better Training Set for Optical Flow (2021, methodology): “RenderGAN [42] learns to augment the dataset for handwriting classification.”
Improving Augmentation and Evaluation Schemes for Semantic Image Synthesis (2020, methodology): “While in principle augmentation strategies can be applied to any class of models, their utility in improving the training of generative adversarial networks (GANs) for image synthesis has not been widely studied, and very little attention has been paid to developing novel augmentation strategies spe”
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation (2020, methodology): “Unlike the hand-crafted approaches such as horizontal ﬂip-ping and random cropping, several recent approaches have been proposed to learn the data augmentation (Cubuk et al., 2019; DeVries & Taylor, 2017a; Lemley et al., 2017; Perez & Wang, 2017; Sixt et al., 2018; Tran et al., 2017).”