Domain randomization for transferring deep neural networks from simulation to the real world

By Joshua Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, Pieter Abbeel. 2017

3,883 citations recorded by the source corpus.

Bridging the ‘reality gap’ that separates simulated robotics from experiments on hardware could accelerate robotic research through improved data availability. This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator. With enough variability in the simulator, the real world may appear to the model as just another variation. We focus on the task of object localization, which is a stepping stone to general robotic manipulation skills. We find that it is possible to train a real-world object detector that is accurate to 1.5 cm and robust to distractors and partial occlusions using only data from a simulator with non-realistic random textures. To demonstrate the capabilities of our detectors, we show they can be used to perform grasping in a cluttered environment. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images (without pre-training on real images) to the real world for the purpose of robotic control.

Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

  1. Jitendra Malik
  2. Sergey Levine
  3. Silvio Savarese
  4. Abhinav Kumar Gupta
  5. Dieter Fox
  6. Kristen Grauman
  7. Pieter Abbeel
  8. Andreas Geiger

In Artificial Intelligence

  1. Sergey Levine
  2. Pieter Abbeel
  3. Yoshua Bengio
  4. Chelsea Finn
  5. Mohit Bansal
  6. Ruslan Salakhutdinov
  7. Trevor Darrell
  8. Quoc V. Le

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Jitendra Malik
  5. Xiaogang Wang
  6. Ming-Hsuan Yang
  7. Andrew Zisserman
  8. Andrea Vedaldi

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Chelsea Finn
  4. Ofir Nachum
  5. Shixiang Shane Gu
  6. Shimon Whiteson
  7. Abhishek Gupta
  8. Vikash Kumar

Citations in the corpus