Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping

By Konstantinos Bousmalis, A. Irpan, Paul Wohlhart, Yunfei Bai, Matthew Kelcey, Mrinal Kalakrishnan, Laura Downs, Julian Ibarz, Peter Pastor, Kurt Konolige and 2 more. 2018

731 citations recorded by the source corpus.

Instrumenting and collecting annotated visual grasping datasets to train modern machine learning algorithms can be extremely time-consuming and expensive. An appealing alternative is to use off-the-shelf simulators to render synthetic data for which ground-truth annotations are generated automatically. Unfortunately, models trained purely on simulated data often fail to generalize to the real world. We study how randomized simulated environments and domain adaptation methods can be extended to train a grasping system to grasp novel objects from raw monocular RGB images. We extensively evaluate our approaches with a total of more than 25,000 physical test grasps, studying a range of simulation conditions and domain adaptation methods, including a novel extension of pixel-level domain adaptation that we term the GraspGAN. We show that, by using synthetic data and domain adaptation, we are able to reduce the number of real-world samples needed to achieve a given level of performance by up to 50 times, using only randomly generated simulated objects. We also show that by using only unlabeled real-world data and our GraspGAN methodology, we obtain real-world grasping performance without any real-world labels that is similar to that achieved with 939,777 labeled real-world samples.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Representation & Self-Supervised Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Representation & Self-Supervised Learning

In Robotics & Control

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, methodology): “Our approach may complement sim2real approaches outlined in previous work [4, 33].”
World Models (2018, methodology): “Our approach may complement sim2real approaches outlined in (Bousmalis et al., 2017; Higgins et al., 2017).”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “Both feature-level and pixel-level domain adaptation are combined in [122], where the results indicate that including simulated data can improve the vision-based grasping system, achieving comparable performance with 50 times fewer real-world samples.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “For manipulation tasks, Bousmalis et al. (2017) deal with the reality gap by adapting synthetic images to the realistic domain before feeding them into the DRL policy network during the training phase.”
A Framework for Data-Driven Robotics (2019, background): “Sim2real: The sim2real framework aims to seamlessly transfer policies trained in simulated environments to the real world [29]–[33].”
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, methodology): “resentations of real and synthetic data [16], using generative models to make synthetic images more realistic [17], fine-tuning the policies trained in simulation in the real world [18], learning inverse dynamics models [19], multiobjective optimization of task fitness and transferability to the rea”
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots (2018, background): “feature level [49, 12] or pixel level [4] to transfer the controller.”
RLBench: The Robot Learning Benchmark & Learning Environment (2019, background): “c) Sim-to-Real Transfer: Recently there has been a large amount of work in learning control policies in simulation and then transferring these to the real world [48], [49], [50], [51], [52], [53].”
Reinforcement and Imitation Learning for Diverse Visuomotor Skills (2018, background): “[2] augment the training with simulated data to learn grasp prediction of diverse shapes.”
CyCADA: Cycle-Consistent Adversarial Domain Adaptation (2017, methodology): “Bousmalis et al. (2017b) use a content similarity loss to ensure the generated target image is similar to the original source image; however, this requires prior knowledge about which parts of the image stay the same across domains (e.g. foreground).”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, methodology): “[3], introduced the GraspGAN method, which combines pixel-level with feature-level domain adaptation to limit the amount of real data needed for learning grasping.”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “By using synthetic data and domain adaptation [237], the number of real-world examples needed to achieve a given level of performance is reduced by up to 50 times, utilizing only randomly generated simulated objects.”
Implicit 3D Orientation Learning for 6D Object Detection from RGB Images (2018, methodology): “Generative Adversarial Networks (GANs) have been deployed for unsupervised DA by generating realistic from synthetic images to train classifiers [33], 3D pose estimators [3] and grasping algorithms [2].”
A Review of Domain Adaptation without Target Labels (2019, neutral): “stationary robots [54].”
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control (2018, background): “Most prior approaches to domain adaptation either train domain invariant representation [20, 4, 11, 3], learn a representation transformation from one domain to another via transfer learning [12, 34, 29], or employ domain randomization in simulation [30, 33] which produces robust models that can gen”
A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “These techniques can be employed in the domain adaptation methods that utilize GANs [20, 21, 41, 96, 143, 160, 208, 219, 245, 250].”
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives (2022, methodology): “Following AdaBN, several works have shown that the BN alignment can be added on top of other UDA methods (Li, 2018; Bousmalis et al., 2018; French et al., 2017; Kang et al., 2019).”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, methodology): “For many tasks, it is most natural to formulate them as binary reward tasks (Irpan 2018): a grasping robot can either succeed or fail at grasping an object, a pouring robot can pour water into a glass or not, and a mobile robot can reach the destination or not.”