Domain randomization for transferring deep neural networks from simulation to the real world
By Joshua Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, Pieter Abbeel. 2017
3,883 citations recorded by the source corpus.
Bridging the ‘reality gap’ that separates simulated robotics from experiments on hardware could accelerate robotic research through improved data availability. This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator. With enough variability in the simulator, the real world may appear to the model as just another variation. We focus on the task of object localization, which is a stepping stone to general robotic manipulation skills. We find that it is possible to train a real-world object detector that is accurate to 1.5 cm and robust to distractors and partial occlusions using only data from a simulator with non-realistic random textures. To demonstrate the capabilities of our detectors, we show they can be used to perform grasping in a cluttered environment. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images (without pre-training on real images) to the real world for the purpose of robotic control.
Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Jitendra Malik
- Sergey Levine
- Silvio Savarese
- Abhinav Kumar Gupta
- Dieter Fox
- Kristen Grauman
- Pieter Abbeel
- Andreas Geiger
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Yoshua Bengio
- Chelsea Finn
- Mohit Bansal
- Ruslan Salakhutdinov
- Trevor Darrell
- Quoc V. Le
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Jitendra Malik
- Xiaogang Wang
- Ming-Hsuan Yang
- Andrew Zisserman
- Andrea Vedaldi
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Ofir Nachum
- Shixiang Shane Gu
- Shimon Whiteson
- Abhishek Gupta
- Vikash Kumar
Citations in the corpus
- Shortcut learning in deep neural networks (2020, background): “helps a lot in closing this gap [70, 71].”
- Scalable agent alignment via reward modeling: a research direction (2018, methodology): “In the absence of such a solution we could rely on out-of-distribution detection to be able to defer to a human operator or widening the training distribution to encompass all relevant cases (Tobin et al., 2017).”
- DisCoRL: Continual Reinforcement Learning via Policy Distillation (2019, methodology): “One of these approaches is Domain Randomization [Tobin et al., 2017], which we use in this paper.”
- Unsupervised state representation learning with robotic priors: a robustness benchmark (2017, background): “• Dataset 4: Colorful75 : We added domain randomization [22], i.e., every object and the table changes color from sequence to sequence.”
- Simulation-assisted machine learning (2018, methodology): “Varying the simulation conditions during training, called domain randomization, is used to improve model generalization (Tobin et al., 2017).”
- On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Moreover, sim2real transfer is of major interest in the robotic learning community, because of limited data and supervision in the real world (Tobin et al., 2017; Rusu et al., 2017; Peng et al., 2018; James et al., 2019; Yan et al., 2020; Andrychowicz et al., 2020).”
- Solving Rubik's Cube with a Robot Hand (2019, methodology): “In [77], we were able to train a control policy and a vision model in simulation and then transfer both to a real robot through the use of domain randomization [106, 80].”
- A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Tobin et al. (2017) proposed to randomise the lighting conditions, viewing angles and textures of objects during the training phase of the DRL policies in simulation, in the hope that after being exposed to enough variations, the learned model can naturally generalize to real-world scenarios.”
- A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “However, Sim2Real [27]–[29] and robotics in the physical world will not be covered in this paper.”
- The limits and potentials of deep learning for robotics (2018, background): “Another promising recent direction is domain randomization [97], [118].”
- RoboNet: Large-Scale Multi-Robot Learning (2019, background): “A number of works have studied learning representations and policies that transfer across domains, including transfer from simulation to the real world [1, 33, 34], transfer across different dynamics [35, 36, 37, 2], transfer across robot morphologies with invariant feature spaces [38] and modularit”
- A Framework for Data-Driven Robotics (2019, background): “Even with careful simulation design there remains a domain gap between simulation and reality that needs to be dealt with [30], [34].”
- Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, methodology): “As it was shown in [6, 7, 9], it is possible to design a distribution of simulation parameters p (⇠), such that a policy trained on P⇠⇠p would perform well on a real world dynamics distribution.”
- Sim-to-Real: Learning Agile Locomotion For Quadruped Robots (2018, methodology): “A robust controller can be learned by injecting noise to the simulation [19], adding perturbations to the simulated robot [37], leveraging multiple simulators [3], using domain randomization [48] and dynamics randomization [29, 40, 34].”
- Visual Imitation Made Easy (2020, background): “It has also been extensively used in context of RL, where domain randomization is proposed to transfer learning from simulation to real world [36, 37, 38].”
- Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels (2020, neutral): “There are also domain randomization techniques [52, 49] which synthetically apply similar variations, but assume control of the data generation procedure, in contrast to our method.”
- Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization (2021, background): “RR is conceptually related to domain randomization (Tobin et al., 2017) with the difference that we train separate policies instead of a single universal one, which suffers from mode collapse (see appendix D.2.3).”