Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks

By Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, A. Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, Konstantinos Bousmalis. 2018

508 citations recorded by the source corpus.

Real world data, especially in the domain of robotics, is notoriously costly to collect. One way to circumvent this can be to leverage the power of simulation to produce large amounts of labelled data. However, training models on simulated images does not readily transfer to real-world ones. Using domain adaptation methods to cross this "reality gap" requires a large amount of unlabelled real-world data, whilst domain randomization alone can waste modeling power. In this paper, we present Randomized-to-Canonical Adaptation Networks (RCANs), a novel approach to crossing the visual reality gap that uses no real-world data. Our method learns to translate randomized rendered images into their equivalent non-randomized, canonical versions. This in turn allows for real images to also be translated into canonical sim images. We demonstrate the effectiveness of this sim-to-real approach by training a vision-based closed-loop grasping reinforcement learning agent in simulation, and then transferring it to the real world to attain 70% zero-shot grasp success on unseen objects, a result that almost doubles the success of learning the same task directly on domain randomization alone. Additionally, by joint finetuning in the real-world with only 5,000 real-world grasps, our method achieves 91%, attaining comparable performance to a state-of-the-art system trained with 580,000 real-world grasps, resulting in a reduction of real-world data by more than 99%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Moreover, sim2real transfer is of major interest in the robotic learning community, because of limited data and supervision in the real world (Tobin et al., 2017; Rusu et al., 2017; Peng et al., 2018; James et al., 2019; Yan et al., 2020; Andrychowicz et al., 2020).”
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “Some fields have been able to narrow the gap between simulation and reality [56, 5, 23], which has led to remarkable achievements (e.”
RoboNet: Large-Scale Multi-Robot Learning (2019, background): “…works have studied learning representations and policies that transfer across domains, including transfer from simulation to the real world [1, 33, 34], transfer across different dynamics [35, 36, 37, 2], transfer across robot morphologies with invariant feature spaces [38] and modularity [39],…”
RLBench: The Robot Learning Benchmark & Learning Environment (2019, background): “Recently there has been a large amount of work in learning control policies in simulation and then transferring these to the real world [50]–[55].”
Off-Policy Evaluation via Off-Policy Classification (2019, methodology): “• RCAN: The RCAN model, as described in [12], trained in simulation with a pixel level adaptation model.”
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning (2022, background): “This includes works that generalize skills to novel objects [31, 32, 33, 34, 35], to novel 76 environments [36], from simulation to reality [37, 38, 39, 40, 41], and to new manipulation skills 77 and objects [42, 43, 44, 45].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, unknown): “A novel extension of pixel-level domain adaptation named GraspGAN [237] was proposed for robotic grasping [238], [239].”
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes (2020, unknown): “It is standard practice for robot learning to partially randomize the environment’s parameters in order to make the policy more robust [56, 57, 58, 59].”
TriFinger: An Open-Source Robot for Learning Dexterity (2020, unknown): “However, results obtained in simulation experiments do often not translate to real systems (e.g. Tobin et al., 2017; James et al., 2019).”
Synthetic Data for Deep Learning (2019, background): “[279] consider the same kind of approach for robotic grasping.”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, methodology): “An alternative approach is that of James et al. (2019) which trains an adapter network to convert real-world images to canonical simulation images, allowing a policy trained only in simulation to be applied in the real-world.”
RetinaGAN: An Object-aware Approach to Sim-to-Real Transfer (2020, methodology): “GANs are a commonly applied method for pixellevel transfer which only require unpaired images from both domains [18], [19], [5], [20], [21].”
Understanding Continual Learning Settings with Data Distribution Drift Analysis (2021, background): “For example, in robotics, many approaches are focused on transferring a model trained in a simulated environment to real environments (Peng et al., 2018; Traor ´ e et al., 2019; James et al., 2019).”
Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real (2019, background): “For this reason, we avoid fine-tuning in the real world [30, 31], train wholly in simulation, and perform a zero-shot transfer to the real world.”
Sim2real transfer learning for 3D human pose estimation: motion to the rescue (2019, background): “In some cases, these abstractions can be obtained automatically from a simulator alone [31].”
MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation (2021, unknown): “UDA for semantic segmentation is a crucial practical problem since it may be beneﬁcial for various real-world applications, such as simulation for robots [18] and autonomous driving [44].”
The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints (2018, methodology): “Both the dataset and HyperTrees might also be useful for developing, evaluating and comparing algorithms utilizing sim-to-real transfer, GANs, domain adaptation, and metalearning[26], [27].”
Neurosymbolic Programming (2021, background): “For instance, in sequential decision making scenarios (such as those typically solved using reinforcement learning), the training environments may be substantially different from the test environments (James et al., 2019; Liu et al., 2020).”