Synthetic Data for Deep Learning

By Sergey I. Nikolenko. 2019

471 citations recorded by the source corpus.

Synthetic data is an increasingly popular tool for training deep learning models, especially in computer vision but also in other areas. In this work, we attempt to provide a comprehensive survey of the various directions in the development and application of synthetic data. First, we discuss synthetic datasets for basic computer vision problems, both low-level (e.g., optical flow estimation) and high-level (e.g., semantic segmentation), synthetic environments and datasets for outdoor and urban scenes (autonomous driving), indoor scenes (indoor navigation), aerial navigation, simulation environments for robotics, applications of synthetic data outside computer vision (in neural programming, bioinformatics, NLP, and more); we also survey the work on improving synthetic data development and alternative ways to produce it such as GANs. Second, we discuss in detail the synthetic-to-real domain adaptation problem that inevitably arises in applications of synthetic data, including synthetic-to-real refinement with GAN-based models and domain adaptation at the feature/model level without explicit data transformations. Third, we turn to privacy-related applications of synthetic data and review the work on generating synthetic datasets with differential privacy guarantees. We conclude by highlighting the most promising directions for further work in synthetic data studies.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Signal Processing

Citations in the corpus

Occluded Video Instance Segmentation: A Benchmark (2021, background): “2014) with partial occlusions, and some works (Nikolenko, 2019; Kar et al., 2019; Devaranjan et al., 2020) synthesize structured amodal data in occluded scenes using simulators.”
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding (2020, background): “rstanding research because they can be rendered very efﬁciently. However, these datasets and environments introduce a large domain gap between real and synthetic images that must be handled carefully [51]. In contrast, our dataset aims to be as photorealistic as possible, thereby signiﬁcantly reduci”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, methodology): “data [194] is an increasingly popular tool for training deep models, and offers promising opportunities to be explored further for FGIA.”