Learning to push by grasping: Using multiple tasks for effective learning

By Lerrel Pinto, Abhinav Kumar Gupta. 2016

142 citations recorded by the source corpus.

Recently, end-to-end learning frameworks are gaining prevalence in the field of robot control. These frameworks input states/images and directly predict the torques or the action parameters. However, these approaches are often critiqued due to their huge data requirements for learning a task. The argument of the difficulty in scalability to multiple tasks is well founded, since training these tasks often require hundreds or thousands of examples. But do end-to-end approaches need to learn a unique model for every task? Intuitively, it seems that sharing across tasks should help since all tasks require some common understanding of the environment. In this paper, we attempt to take the next step in data-driven end-to-end learning frameworks: move from the realm of task-specific models to joint learning of multiple robot tasks. In an astonishing result we show that models with multi-task learning tend to perform better than task-specific models trained with same amounts of data. For example, a deep-network learned with 2.5K grasp and 2.5K push examples performs better on grasping than a network trained on 5K grasp examples.

Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Multi-task Self-Supervised Visual Learning (2017, methodology): “Some previous self-supervised learning literature has suggested performance gains from combining self-supervised tasks [32, 44], although these works used relatively similar tasks within relatively restricted domains where extra information was provided besides pixels.”
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning (2018, neutral): “More recent data-driven methods explore the prospects of training model-agnostic deep grasping policies [26], [7], [27], [9], [10], [8] that detect grasps by exploiting learned visual features, and without explicitly using object specific knowledge (i.”
Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning (2017, background): “Recent results have shown success on games (Mnih et al. (2013)), locomotion (Schulman et al. (2015)), and a variety of robotic manipulation skills (Pinto & Gupta (2017); Schulman et al. (2016); Gu et al. (2017)).”
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, background): “Most of the existing robotics datasets focus on a single task in isolation such as grasping Pinto and Gupta [2], Levine et al. [3], pushing [5, 6], poking [4] or knot-tying [7].”
Hindsight Experience Replay (2017, methodology): “4.3 for details) and therefore our approach may be applicable even if there is only one task we would like the agent to perform (a similar situation was recently observed by Pinto and Gupta (2016)).”
Asymmetric Actor Critic for Image-Based Robot Learning (2017, background): “The tasks of Forward Pushing and Block Move are similar to pushing objects [31], [32], however as in the case of Picking , this paper focuses on the learning of the ﬁne pushing behaviour.”
Learning to fly by crashing (2017, background): “Self supervised methods [6], [7], [18], [19], show how large scale data collection in the real world can be used to learn tasks like grasping and pushing objects in a tabletop environment.”
Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration (2017, methodology): “It is also shown that sharing the parameters of a neural network among different tasks not only improves the results, but also it is even better to train the model using the data of multiple related tasks instead of using the same amount of data from the original task [26].”
Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias (2018, background): “Agarwal et al. [8], Finn et al. [9], and Pinto and Gupta [26] collected data of a manipulator pushing objects on a table.”
BridgeData V2: A Dataset for Robot Learning at Scale (2023, background): “Several prior projects have assembled large robot datasets for a single behavior like grasping [19, 20, 21, 22, 23], poking [24], pushing [25, 26], or rope manipulation [27].”
CASSL: Curriculum Accelerated Self-Supervised Learning (2017, methodology): “Instead, we employ a shared architecture commonly used in multi-task learning [23], [32], such that the non-linear relationship between the different parameters could be learned.”
Learning Actionable Representations from Visual Observations (2018, background): “Multi-task learning has also been shown to be helpful in learning more robust internal representations that ultimately lead to improvement in performance [28], [29], [30].”
Learning Visual Affordances for Robotic Manipulation (2019, background): “[89] improve the performance of these deep policies by using models pre-trained on auxiliary tasks such as poking.”
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale (2021, methodology): “Prior work indicates that multi-task RL can indeed amortize the cost of single-task learning [20, 56, 60, 80, 30].”
What Should Not Be Contrastive in Contrastive Learning (2020, background): “This phenomenon becomes more obvious in multi-task self-supervised learning (Doersch & Zisserman, 2017; Wang et al., 2017; Pinto & Gupta, 2017; Piergiovanni et al., 2020; Alwassel et al., 2019) as the optimization goal for each task can be very different depending on the pretext task.”
Video Jigsaw: Unsupervised Learning of Spatiotemporal Context for Video Action Recognition (2018, background): “Other self-supervised learning tasks on videos use cues like ego-motion [63, 22, 1] as a supervisory signal and other modalities beyond raw pixels such as audio [37, 36] and robot motion [2, 40, 41, 42].”
Cross-Domain Self-Supervised Multi-task Feature Learning Using Synthetic Imagery (2017, background): “Multi-task learning Multi-task learning [10] has been used for a variety of vision problems including surface normal and depth prediction [16, 17], semantic segmentation [45], pose estimation [24], robot manipulation [55, 54], and face detection [77].”
GPLAC: Generalizing Vision-Based Robotic Skills Using Weakly Labeled Images (2017, background): “…the success of deep learning meth-ods has prompted considerable recent interest in learning sensorimotor policies that directly map image pixels to actions [52, 27, 1, 29, 45, 43, 24, 37, 38], following on classic work on end-to-end imitation and reinforcement learning from the 1980s and 90s [40].”
Grasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost Demonstrations (2019, background): “More recent data-driven methods explore the prospects of training object-agnostic grasping policies that detect grasps by exploiting learned visual features, without explicitly using object-specific knowledge [26, 23, 24, 8, 18, 37, 17, 21, 7].”