Towards Adapting Deep Visuomotor Representations from Simulated to Real Environments

By Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Xingchao Peng, Sergey Levine, Kate Saenko, Trevor Darrell. 2015

105 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

One-Shot Imitation Learning (2017, background): “Success stories include domain adaptation in computer vision [64, 34, 28, 4, 15, 24, 33, 59, 14] and control [60, 45, 46, 20, 54].”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Model-based DRL methods are also utilized by Finn et al. (2016) and Tzeng et al. (2015), learning useful state representations for generating successful control policies.”
The limits and potentials of deep learning for robotics (2018, background): “In [70], a ‘confusion’ loss is proposed which forces the model to ignore variations in the data that separate the two domains [122], [123], and [121] attempts to address the simulation to reality gap by using aligned data.”
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, methodology): “to account for potential domain shifts between observations [16, 39, 40].”
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots (2018, background): “feature level [49, 12] or pixel level [4] to transfer the controller.”
(CAD)$^2$RL: Real Single-Image Flight without a Single Real Image (2016, neutral): “In contrast to prior work on domain adaptation [35, 41], our method does not require any real images during training.”
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks (2016, background): “Consequently, there has been a renewed interest in training models in the synthetic domain and applying them in real–world settings [8, 48, 38, 43, 25, 32, 35, 37].”
Third-Person Imitation Learning (2017, background): “The work most closely related to ours is Tzeng et al. (2014; 2015), who also consider an explicit domain confusion loss, forcing trained classiﬁers to rely on features that don’t allow to distinguish between two domains.”
The Curious Robot: Learning Visual Representations via Physical Interactions (2016, background): “There has been a recent trend to use deep networks in robotics systems for grasp regression [25,35,36] or learning policies for a variety of tasks [37,38].”
A Brief Survey of Deep Reinforcement Learning (2017, neutral): “(c) Utilising the potentially unlimited amount of training data that can be amassed in robotic simulators, several methods aim to transfer knowledge from the simulator to the real world [18, 97, 124].”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, background): “However, DA techniques can further help to adjust the model to the target real data especially when no or few labeled examples are available in the real domain [230, 252, 211].”
Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model (2016, methodology): “Our approach is agnostic to how the policy πsource was obtained.”
Adapting Deep Visuomotor Representations with Weak Pairwise Constraints (2015, methodology): “ estimation for a real robotic task using minimal real world data, suggesting that adaption from simulation to the real world can be eective for robotic learning. In an earlier version of this paper [36], we demonstrated initial results using domain confusion constraints on PR2 visuomotor policies b”
3D Simulation for Robot Arm Control with Deep Q-Learning (2016, neutral): “For transfer of policies from simulation to a real robot, a cross-domain loss was proposed in [18] to incorporate both simulated and real-world data within the same loss function.”
Robust Visual Domain Randomization for Reinforcement Learning. (2019, background): “L G ] 6 M ar 2 02 0 the other hand, whereas the notion of learning similar state representations to improve transfer to new domains is not new Tzeng et al. (2015); Gupta et al. (2017); Daftry et al. (2016), previous work has focused on cases when simulated and target domains are both known, and…”