Combining self-supervised learning and imitation for vision-based rope manipulation

By Ashvin Nair, Dian Chen, Pulkit Agrawal, Phillip Isola, Pieter Abbeel, Jitendra Malik, Sergey Levine. 2017

338 citations recorded by the source corpus.

Manipulation of deformable objects, such as ropes and cloth, is an important but challenging problem in robotics. We present a learning-based system where a robot takes as input a sequence of images of a human manipulating a rope from an initial to goal configuration, and outputs a sequence of actions that can reproduce the human demonstration, using only monocular images as input. To perform this task, the robot learns a pixel-level inverse dynamics model of rope manipulation directly from images in a self-supervised manner, using about 60K interactions with the rope collected autonomously by the robot. The human demonstration provides a high-level plan of what to do and the low-level inverse model is used to execute the plan. We show that by combining the high and low-level plans, the robot can successfully manipulate a rope into a variety of target shapes using only a sequence of human-provided images for direction.

Subfields: Aerospace Engineering, Biomedical Engineering, Control and Systems Engineering, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Biomedical Engineering

In Control and Systems Engineering

In Robotics & Control

Citations in the corpus

Accelerating Online Reinforcement Learning with Offline Datasets (2020, methodology): “Other prior works have incorporated demonstration data into the replay buffer using off-policy RL methods [45, 29].”
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control (2018, background): “Several prior works have also sought to learn inverse models that map from pairs of observations to actions, which can then be used greedily to carry out short-horizon tasks [20], [21].”
Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, background): “Then, the inferred actions can be executed to reproduce the trainers states [Nair et al. , 2017].”
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, background): “Most of the existing robotics datasets focus on a single task in isolation such as grasping Pinto and Gupta [2], Levine et al. [3], pushing [5, 6], poking [4] or knot-tying [7].”
One-Shot Visual Imitation Learning via Meta-Learning (2017, background): “However, this requires additional robot experience to optimize the reward [19, 20, 21].”
Model-Ensemble Trust-Region Policy Optimization (2018, methodology): “…required a combination with model-free techniques (Oh et al., 2015; Heess et al., 2015; Nagabandi et al., 2017), or domain-speciﬁc policy learning or planning algorithms (Lenz et al., 2015; Agrawal et al., 2016; Pinto & Gupta, 2016; Levine et al., 2016b; Finn & Levine, 2017; Nair et al., 2017).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “To deal with high-dimensional data such as raw images, a deep learning approach is employed for modeling a forward dynamics in the most recent studies [Oh et al., 2015, Finn et al., 2017a, Baram et al., 2017, Nair et al., 2017].”
Learning Latent Plans from Play (2019, background): “Goal conditioned policies have been explored extensively in the literature for reinforcement learning [23, 44, 36, 52, 3, 32, 48, 8, 58], as well as for control via inverse models [1, 35, 9, 61].”
Zero-Shot Visual Imitation (2018, methodology): “The idea of learning goal-conditioned policies has been explored in (Agrawal et al., 2016; Andrychowicz et al., 2017; Nair et al., 2017; Schaul et al., 2015).”
Learning Plannable Representations with Causal InfoGAN (2018, methodology): “[29] of a robot randomly poking a rope.”
Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias (2018, methodology): “Similarly, Nair et al. [10] collects data for manipulating a rope on a table while Yahya et al. [27] used several robots in parallel to train a policy to open a door.”
Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation (2018, background): “In the area of self-supervised visual robot learning, while some recent work has sought to understand ‘how will the world change given the robot’s action?” [16, 17] in this work we instead ask “what is the current visual state of the robot’s world?”.”
Synthesizing Environment-Aware Activities via Activity Sketches (2019, background): “For example, [18, 11, 1] focus on learning to navigate in environments or manipulate objects, while in visual imitation learning, multiple works have used videos to learn to manipulate objects under low supervision regimes [14, 7] or imitate kinetic human behaviors [12].”
Time Reversal as Self-Supervision (2018, background): “Similarly, Agarwal et al. [27] and Nair et al. [28] have learned inverse dynamics models in image space.”
Modeling, learning, perception, and control methods for deformable object manipulation (2021, background): “Trajectories may be generated on the basis of human demonstrations (92), and a plan represented in some latent (6, 90, 91) or original (89) state spaces.”
Imitation Learning: A Survey of Learning Methods, Environments and Metrics (2024, neutral): “Nair et al. [10] use this form of learning to teach a robot how to manipulate a rope in different ways.”
Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation (2019, methodology): “These methods have shown successes in many control tasks given visual inputs, such as pushing objects in a cluttered tray [7] and manipulating deformable objects [9].”
Challenges and Outlook in Robotic Manipulation of Deformable Objects (2021, methodology): “Vision is used in tasks such as rope manipulation [25], [26] or cloth unfolding [27], [28], where the object exhibits large global deformation.”