Adapting Deep Visuomotor Representations with Weak Pairwise Constraints

By Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Pieter Abbeel, Sergey Levine, Kate Saenko, Trevor Darrell. 2015

143 citations recorded by the source corpus.

Real-world robotics problems often occur in domains that differ significantly from the robot's prior training environment. For many robotic control tasks, real world experience is expensive to obtain, but data is easy to collect in either an instrumented environment or in simulation. We propose a novel domain adaptation approach for robot perception that adapts visual representations learned on a large easy-to-obtain source dataset (e.g. synthetic images) to a target real-world domain, without requiring expensive manual data annotation of real world data before policy search. Supervised domain adaptation methods minimize cross-domain differences using pairs of aligned images that contain the same object or scene in both the source and target domains, thus learning a domain-invariant representation. However, they require manual alignment of such image pairs. Fully unsupervised adaptation methods rely on minimizing the discrepancy between the feature distributions across domains. We propose a novel, more powerful combination of both distribution and pairwise image alignment, and remove the requirement for expensive annotation by using weakly aligned pairs of images in the source and target domains. Focusing on adapting from simulation to real world data using a PR2 robot, we evaluate our approach on a manipulation task and show that by using weakly paired images, our method compensates for domain shift more effectively than previous techniques, enabling better robot performance in the real world.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Domain randomization for transferring deep neural networks from simulation to the real world (2017, neutral): “In [49], the authors learn a correspondence”
One-Shot Imitation Learning (2017, background): “Success stories include domain adaptation in computer vision (Yang et al., 2007; Kulis et al., 2011; Tzeng et al., 2014; Donahue et al., 2014) and control (Tzeng et al., 2015; Rusu et al., 2016; Sadeghi & Levine, 2016; Gupta et al., 2017; Stadie et al., 2017).”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Model-based DRL methods are also utilized by Finn et al. (2016) and Tzeng et al. (2015), learning useful state representations for generating successful control policies.”
DARLA: Improving Zero-Shot Transfer in Reinforcement Learning (2017, background): “, 2016) has not yet translated as well to reality, despite various attempts (Tobin et al., 2017; Tzeng et al., 2016; Daftry et al., 2016; Finn et al., 2015; Rusu et al., 2016).”
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization (2017, background): “Learning invariant features has emerged as a promising approach of taking advantage of these commonalities [11],”
Third-Person Imitation Learning (2017, background): “The work most closely related to ours is Tzeng et al. (2014; 2015), who also consider an explicit domain confusion loss, forcing trained classiﬁers to rely on features that don’t allow to distinguish between two domains.”
Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task (2017, methodology): “To overcome human involvement, GPS has been used with domain adaptation of both simulated and real world images to map to a common feature space for putting a rope around a hook [22]; though this still requires images from the real world.”
Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics (2017, background): “Methods to transfer representations from simulation to reality include finetuning the higher network layers [67], domain confusion [57], or progressive networks [51].”
End-to-End Learning of Driving Models from Large-Scale Video Datasets (2016, background): “, as was done in [17], and more recently in [23, 20, 3]).”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, methodology): “In the context of robotics, visual domain adaptation has also been used for simulation-to-real-world transfer [62, 56, 3].”
Self-Similarity Grouping: A Simple Unsupervised Cross Domain Adaptation Approach for Person Re-Identification (2018, background): “Recently, some works solve this problem by mapping the source data and target data to the same feature space for the domain-invariant representations [16, 17, 27, 37].”
Sim2Real View Invariant Visual Servoing by Recurrent Control (2017, background): “Most prior approaches to domain adaptation either train domain invariant representation [20, 4, 11, 3], learn a representation transformation from one domain to another via transfer learning [12, 34, 29], or employ domain randomization in simulation [30, 33] which produces robust models that can gen”
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation (2017, background): “Investigating domain shift is left for future work, and could be done, for example, using domain adaptation [40].”
An Introduction to Deep Reinforcement Learning (2018, unknown): “Another approach to zero-shot transfer is to use algorithms that enforce states that relate to the same underlying task but have different renderings to be mapped into an abstract state that is close (Tzeng et al., 2015; François-Lavet et al., 2018).”
The Ingredients of Real-World Robotic Reinforcement Learning (2020, background): “Much of the focus in prior work has been on improving the efficiency and generalization of RL algorithms to make real-world training feasible, or else on utilizing simulation and transferring policies into the real world (Tzeng et al., 2015; Tobin et al., 2017; Peng et al., 2017; Clavera et al., 201”