Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

By Lerrel Pinto, Abhinav Kumar Gupta. 2015

1,220 citations recorded by the source corpus.

Current model free learning-based robot grasping approaches exploit human-labeled datasets for training the models. However, there are two problems with such a methodology: (a) since each object can be grasped in multiple ways, manually labeling grasp locations is not a trivial task; (b) human labeling is biased by semantics. While there have been attempts to train robots using trial-and-error experiments, the amount of data used in such experiments remains substantially low and hence makes the learner prone to over-fitting. In this paper, we take the leap of increasing the available training data to 40 times more than prior work, leading to a dataset size of 50K data points collected over 700 hours of robot grasping attempts. This allows us to train a Convolutional Neural Network (CNN) for the task of predicting grasp locations without severe overfitting. In our formulation, we recast the regression problem to an 18-way binary classification over image patches. We also present a multi-stage learning approach where a CNN trained in one stage is used to collect hard negatives in subsequent stages. Our experiments clearly show the benefit of using large-scale datasets (and multi-stage training) for the task of grasping. We also compare to several baselines and show state-of-the-art performance on generalization to unseen objects for grasping.

Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “One particularly important domain for self-supervised learning is robotics, where representations can be learned by interacting with the environment [2, 53, 52, 60], and where success greatly depends on making effective use of unlabeled data.”
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, background): “In order to overcome the bottleneck, there have been recent efforts on visual representation learning using web-supervision [2, 5, 6, 9, 21, 23, 24, 27] or unsupervised [10, 11, 31, 32, 34, 42, 43] paradigms.”
End-to-End Training of Deep Visuomotor Policies (2015, background): “Many prior robotic applications of CNNs do not directly consider control, but employ CNNs for the perception component of a larger robotic system (Hadsell et al., 2009; Sung et al., 2015; Lenz et al., 2015b; Pinto and Gupta, 2015).”
Self-Supervised GANs via Auxiliary Rotation Loss (2018, methodology): “It has been widely used in the video domain [39, 40], the robotics domain [41, 42] and the image domain [24, 35].”
Multi-task Self-Supervised Visual Learning (2017, background): “In the last few years, a considerable number of such tasks have been proposed [1, 2, 6, 7, 8, 17, 20, 21, 23, 25, 26, 27, 28, 29, 31, 39, 40, 42, 43, 46, 47], such as asking a neural network to colorize grayscale images, ﬁll in image holes, solve jigsaw puzzles made from image patches, or predict…”
Visually Indicated Sounds (2015, methodology): “Our dataset is also related to robotic manipulation datasets [41, 35, 15].”
Generative Image Modeling Using Style and Structure Adversarial Networks (2016, background): “Some examples of these auxiliary tasks include predicting: the relative location of two patches [2], ego-motion in videos [15,16], physical signals [17,18,19].”
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “, 2018) (see Figure 5), as well as more domain-specific algorithms, such as a single-step bandit formulation (Pinto and Gupta, 2016).”
Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning (2017, background): “Deep reinforcement learning has demonstrated success in policy search for tasks in domains like game playing (Mnih et al., 2015; Silver et al., 2016; 2017; Kempka et al., 2016; Mirowski et al., 2017) and robotic control (Levine et al., 2016a;b; Pinto & Gupta, 2016).”
Sim-to-Real Robot Learning from Pixels with Progressive Nets (2016, background): “Alternative approaches embrace big data ideas for robotics ([17, 10]).”
RoboNet: Large-Scale Multi-Robot Learning (2019, background): “Large-scale, self-supervised robot learning approaches have adopted a similar viewpoint [24, 25, 26, 9, 15, 10, 27, 7].”
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control (2018, background): “A number of recent works have studied self-supervised robotic learning, where large-scale unattended data collection is used to learn individual skills such as grasping [22], [23], [24], [25], push-grasp synergies [26], or obstacle avoidance [27], [28].”
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation (2018, background): “Prediction has been an active area of research in computer vision, including prediction of future images [9, 10, 11], motion [12], and even physical events [13, 14, 15].”
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, background): “While there have been recent attempts on learning by collecting large scale data directly on real robots [1, 2, 3, 4], such an approach still remains challenging as collecting real world data is prohibitively laborious and expensive.”
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018, background): “Several works have proposed self-supervised grasping systems [26, 25], but to our knowledge, ours is the first to incorporate long-horizon reasoning via reinforcement learning into a generalizable vision-based system trained on self-supervised real-world data.”
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning (2018, background): “rience [7], [8] or induced by grasp stability metrics [9], [10].”
Deep visual foresight for planning robot motion (2016, background): “Several works have used deep neural networks to process images and represent policies for robotic control, initially for driving tasks [15], [16], later for robotic soccer [17], and most recently for robotic grasping [18], [19] and manipulation [20].”
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection (2016, methodology): “…800,000 grasp attempts on a very large variety of objects, which is more than an order of magnitude larger than prior methods based on direct self-supervision (Pinto & Gupta, 2015) and more than double the dataset size of prior methods based on synthetic grasps from 3D scans (Kappler et al., 2015).”
OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning (2020, background): “…(Silver et al., 2016; Tesauro, 1995; Brown & Sandholm, 2019) and robotics (Levine et al., 2016; Dasari et al., 2019; Peters et al., 2010; Parmas et al., 2019; Pinto & Gupta, 2016; Nachum et al., 2019a), where the agent can act in the environment and sample as many transitions and rewards as needed.”