Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration

By Rouhollah Rahmatizadeh, Pooya Abolghasemi, Ladislau Bölöni, Sergey Levine. 2017

295 citations recorded by the source corpus.

We propose a technique for multi-task learning from demonstration that trains the controller of a low-cost robotic arm to accomplish several complex picking and placing tasks, as well as non-prehensile manipulation. The controller is a recurrent neural network using raw images as input and generating robot arm trajectories, with the parameters shared across the tasks. The controller also combines VAE-GAN-based reconstruction with autoregressive multimodal action prediction. Our results demonstrate that it is possible to learn complex manipulation tasks, such as picking up a towel, wiping an object, and depositing the towel to its previous position, entirely from raw images with direct behavior cloning. We show that weight sharing and reconstruction-based regularization substantially improve generalization and robustness, and training on multiple tasks simultaneously increases the success rate on all tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Robotics & Control

Citations in the corpus

A Framework for Data-Driven Robotics (2019, background): “Demonstrations can also be incorporated through Behavioral Cloning (BC) [27], [28], but this requires high-quality consistent demonstrations of the target task.”
Implicit Behavioral Cloning (2021, methodology): “Explicit “MSE” models are60 trained with Mean Square Error (MSE), explicit “MDN” models are Mixture Density Networks (MDN)61 [21], and implicit “EBM” models are trained with LInfoNCE and optimized with derivative-free optimization.62 Figs.”
Self-Supervised Correspondence in Visuomotor Policy Learning (2019, methodology): “Sometimes polices are learned with a frozen encoder [2], [9], [10], other times in conjunction with end-to-end training [13].”
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation (2017, background): “Using motion capture devices for teleoperation, such as [37], is more intuitive and can solve this issue.”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “dition, although we focused on the trajectory-based approach, recent work such as [Finn et al., 2017b, Nair et al., 2017, Liu et al., 2017, Rahmatizadeh et al., 2017] addressed the problem of generalizing skills based on visual information by using a deep learning approach, which is a promising way…”
Learning Latent Plans from Play (2019, background): “Even single-task imitation models of this sort must contend with multi-modality [46], and goal-conditioned models are typically restricted to short and relatively simple tasks, such as pushing [1], repositioning rope [35], or short-distance navigation [39].”
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning (2018, background): “Most imitation learning and learning from demonstration methods operate at the level of configuration-space trajectories [44, 2], which are typically collected using kinesthetic teaching [32, 1], teleoperation [8, 35, 62], or sensors on the”
Making Efficient Use of Demonstrations to Solve Hard Exploration Problems (2019, methodology): “Behavior Cloning BC is a simple and common baseline method for learning policies from demonstrations (Pomerleau, 1989; Rahmatizadeh et al., 2018).”
Language Conditioned Imitation Learning Over Unstructured Data (2020, methodology): “However, these approaches typically assume that tasks are speciﬁed to the agent at test time via mechanisms like one-hot task selectors ∗ Equal contribution [37, 43], goal images [29, 11, 26], or target conﬁgurations of the state space [14].”
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning (2022, background): “Unlike conditioning on a discrete one-hot task identifier [8], these continuous forms of task 36 specification can in principle enable the robot to generalize zero-shot to new tasks by providing a 37 language or video command of the new task at test time.”
Task-Relevant Adversarial Imitation Learning (2019, background): “BC : BC is effective in solving many control problems [Pomerleau, 1989, Finn et al., 2017, Duan et al., 2017, Rahmatizadeh et al., 2018].”
One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL (2018, methodology): “While BC has problems with accumulating errors over long sequences, it has been used successfully both on its own (Rahmatizadeh et al., 2017) and as an auxiliary loss in combination with RL (Rajeswaran et al., 2017a; Nair et al., 2017b).”
Using Human Gaze to Improve Robustness Against Irrelevant Objects in Robot Manipulation Tasks (2020, background): “The successful applications of behavior cloning to robotics include [6], [13], [26], [27].”
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale (2021, background): “…and composite skills has been addressed through hierarchical reinforcement learning with options [75, 7, 14], combining multiple sub-tasks [6, 16, 49, 26, 69, 89], reusing samples between tasks [39], relabeling experience in hindsight [3], introducing demonstrations [58, 81, 29, 41, 70, 66, 67].”
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation (2024, unknown): “En-hancements to BC include incorporating history with various architectures [12, 47, 59, 77], new training objectives [10, 18, 35, 63, 104], regularization [71], motor primitives [7, 44, 55, 62, 64, 97], and data preprocessing [81].”
Diffusion policy: Visuomotor policy learning via action diffusion (2023, unknown): “The simplest form of explicit policies maps from world state or observation directly to action (Pomerleau, 1988; Zhang et al., 2018; Florence et al., 2020; Ross et al., 2011; Toyer et al., 2020; Rahmatizadeh et al., 2018); Bojarski et al., 2016).”
Transformers for One-Shot Visual Imitation (2020, background): “Prior work [20, 48, 26] showed this limitation matters in practice, and rectiﬁes the situation by predicting a mixture of uni-modal distributions.”
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies (2018, background): “Reinforcement Learning, [76, 50, 71, 45] and its variants like Meta-RL [20, 27, 54, 21, 40, 74, 17, 49] or its sister fields such as imitation learning [1, 26, 22, 89, 37, 63], commonly focus on the last part of the end-to-end active task pipeline: how to choose an action given a “state” from the wo”