VR-Goggles for Robots: Real-to-Sim Domain Adaptation for Visual Control

By Jingwei Zhang, Lei Tai, Peng Yun, Yufeng Xiong, Ming Liu, Joschka Boedecker, Wolfram Burgard. 2018

135 citations recorded by the source corpus.

In this letter, we deal with the reality gap from a novel perspective, targeting transferring deep reinforcement learning (DRL) policies learned in simulated environments to the real-world domain for visual control tasks. Instead of adopting the common solutions to the problem by increasing the visual fidelity of synthetic images output from simulators during the training phase, we seek to tackle the problem by translating the real-world image streams back to the synthetic domain during the deployment phase, to make the robot feel at home. We propose this as a lightweight, flexible, and efficient solution for visual control, as first, no extra transfer steps are required during the expensive training of DRL agents in simulation; second, the trained DRL agents will not be constrained to being deployable in only one specific real-world environment; and third, the policy training and the transfer operations are decoupled, and can be conducted in parallel. Besides this, we propose a simple yet effective shift loss that is agnostic to the downstream task, to constrain the consistency between subsequent frames which is important for consistent policy outputs. We validate the shift loss for artistic style transfer for videos and domain adaptation, and validate our visual control approach in indoor and outdoor robotics experiments.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “Some fields have been able to narrow the gap between simulation and reality [56, 5, 23], which has led to remarkable achievements (e.”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “Similarly, [126] performs domain adaptation to map real world images to simulated images.”
Gibson Env: Real-World Perception for Embodied Agents (2018, background): “per recently reported an evaluation done on a real robot for adaptation using inverse mapping from real images to renderings [97], with positive results.”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, methodology): “This is contrary to other RL methods, such as DDPG [35] and A3C [39], where heavy domain randomization has been shown to cause training to fail [38, 70].”
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning (2022, background): “This includes works that generalize skills to novel objects [31, 32, 33, 34, 35], to novel 76 environments [36], from simulation to reality [37, 38, 39, 40, 41], and to new manipulation skills 77 and objects [42, 43, 44, 45].”
Learning to Drive from Simulation without Real World Labels (2018, background): “Furthermore, the addition of orthogonal, but relevant, approaches on temporal motion consistency [32], [37] could further advance this field.”
Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning (2017, background): “Recent work [22] of real-to-sim domain adaptation for visual control also makes it possible to tackle this problem through raw RGB images.”
Evaluating Real-World Robot Manipulation Policies in Simulation (2024, unknown): “…pipelines that fail to linearly recover the real results despite recovering the correct ranking, and it is overly sensitive to minor noise in evaluations when different policies perform similarly in the real world. evaluations, we want the same comparison to hold in our simulated evaluation.”
Synthetic Data for Deep Learning (2019, methodology): “[690] in a framework called “VR-Goggles for Robots”.”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, methodology): “Conversely, real images can be transformed into simulationlike images via a cGAN [77].”