Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data

By Aayush Prakash, Shaad Boochoon, Mark Brophy, David Acuna, Eric Cameracci, Gavriel State, Omer Shapira, Stan Birchfield. 2018

316 citations recorded by the source corpus.

We present structured domain randomization (SDR), a variant of domain randomization (DR) that takes into account the structure of the scene in order to add context to the generated data. In contrast to DR, which places objects and distractors randomly according to a uniform probability distribution, SDR places objects and distractors randomly according to probability distributions that arise from the specific problem at hand. In this manner, SDR-generated imagery enables the neural network to take the context around an object into consideration during detection. We demonstrate the power of SDR for the problem of 2D bounding box car detection, achieving competitive results on real data after training only on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that SDR outperforms other approaches to generating synthetic data (VKITTI, Sim 200k, or DR), as well as real data collected in a different domain (BDD100K). Moreover, synthetic SDR data combined with real KITTI data outperforms real KITTI data alone.11Video is at http://youtu.be/1WdjWJYx9AY

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Solving Rubik's Cube with a Robot Hand (2019, background): “For vision models, domain randomization has been modified to improve image content diversity [50, 85, 21] and to adapt the distribution by using an adversarial network [120].”
How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies? (2019, result): “Furthermore, our result is consistent with ongoing research in domain randomization for supervised learning which demonstrated the importance of the sampling distribution for sim-to-real transfer success ([28], [29]).”
FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, background): “Domain Randomization (DR) is the common strategy in domain generalization [64, 53, 49, 63, 65, 59, 70, 77, 50] especially when a single source domain is available [64, 70, 77, 50].”
Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation (2020, methodology): “We define scene grammars and represent strings sampled from the grammar as scene graphs following [48,30] (see Fig.”
Synthetic Data for Deep Learning (2019, background): “[462] take the next logical step, continuing this effort to structured domain randomization.”
Learning to Generate Novel Domains for Domain Generalization (2020, unknown): “s, etc. via a computer simulator [49]. Recently, DR has been successfully used in some computer vision applications, such as semantic segmentation [54,55] and vehicle detection for autonomous driving [42]. However, our approach is signicantly dierent from 4 Following [31], homogeneous DG shares the ”
Meta-Sim: Learning to Generate Synthetic Datasets (2019, methodology): “Domain Randomization (DR) is a cheaper alternative to such photorealistic simulation environments [40, 43, 34].”
Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, methodology): “Literature Category Application [27] Augmentation Car detection [78] Augmentation, meta-learning Image classification [30] Augmentation Image classification [26] Augmentation Reinforcement learning [29] Augmentation Reinforcement learning CrossGrad [31] Augmentation, adversarial Font/Image/speech re”
Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data (2019, methodology): “DR has also been utilized to do object detection and 6D pose estimation [49, 36, 45].”