Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection
By Sergey Levine, Peter Pastor, Alex Krizhevsky, Deirdre Quillen. 2016
2,254 citations recorded by the source corpus.
We describe a learning-based approach to hand-eye coordination for robotic grasping from monocular images. To learn hand-eye coordination for grasping, we trained a large convolutional neural network to predict the probability that task-space motion of the gripper will result in successful grasps, using only monocular camera images independent of camera calibration or the current robot pose. This requires the network to observe the spatial relationship between the gripper and objects in the scene, thus learning hand-eye coordination. We then use this network to servo the gripper in real time to achieve successful grasps. We describe two large-scale experiments that we conducted on two separate robotic platforms. In the first experiment, about 800,000 grasp attempts were collected over the course of two months, using between 6 and 14 robotic manipulators at any given time, with differences in camera placement and gripper wear and tear. In the second experiment, we used a different robotic platform and 8 robots to collect a dataset consisting of over 900,000 grasp attempts. The second robotic platform was used to test transfer between robots, and the degree to which data from a different set of robots can be used to aid learning. Our experimental results demonstrate that our approach achieves effective real-time control, can successfully grasp novel objects, and corrects mistakes by continuous servoing. Our transfer experiment also illustrates that data from different robots can be combined to learn more reliable and effective grasping.
Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering, Human-Computer Interaction, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biomedical Engineering
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Andy Zeng
- Jitendra Malik
- Abhinav Kumar Gupta
- Vikash Kumar
- Dieter Fox
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jitendra Malik
- Trevor Darrell
- Dacheng Tao
- Xiaogang Wang
- Li Fei-Fei
- Andrew Zisserman
- Ming-Hsuan Yang
In Control and Systems Engineering
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Dieter Fox
- Abhinav Kumar Gupta
- Vikash Kumar
- Silvio Savarese
- Abhishek Gupta
In Human-Computer Interaction
- Sergey Levine
- Abhinav Kumar Gupta
- Jitendra Malik
- Lerrel Pinto
- Dieter Fox
- Arsalan Mousavian
- Kristen Grauman
- Pieter Abbeel
In Robotics & Control
- Sergey Levine
- Chelsea Finn
- Pieter Abbeel
- Abhinav Kumar Gupta
- Silvio Savarese
- Dieter Fox
- Lerrel Pinto
- Vladlen Koltun
Citations in the corpus
- Data-efficient Deep Reinforcement Learning for Dexterous Manipulation (2017, methodology): “This work thus made use of a complementary solution to the need for large amounts of interaction data: the use of experimental rigs that allow large scale data collection, e.g. [27], including the use of several robots from which experience are gathered in parallel [19, 5, 39].”
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “…prior works have explored offline RL methods for learning robotic grasping, which is a particularly appealing task offline RL methods, since it requires the ability to generalize to a wide range of objects (Pinto and Gupta, 2016; Levine et al., 2018; Kalashnikov et al., 2018; Zeng et al., 2018).”
- Solving Rubik's Cube with a Robot Hand (2019, neutral): “Alternative approaches include using many physical robots simultaneously, in order to be able to collect sufficient experience [36, 57, 49] or leveraging a model-based learning algorithms, which generally possess much more favorable sample complexity characteristics [121].”
- Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning (2017, background): “Deep reinforcement learning has demonstrated success in policy search for tasks in domains like game playing (Mnih et al., 2015; Silver et al., 2016; 2017; Kempka et al., 2016; Mirowski et al., 2017) and robotic control (Levine et al., 2016a;b; Pinto & Gupta, 2016).”
- A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, positive): “• Auxiliary Tasks (Mirowski et al., 2016; Jaderberg et al., 2016; Levine et al., 2016; Yu et al., 2018; Riedmiller et al., 2018): Uses additional supervised or unsupervised tasks (e.g., regressing depth images from color images, detecting loop closures, predicting end-effector poses) alongside the…”
- An Incremental Self-Organizing Architecture for Sensorimotor Learning and Prediction (2017, neutral): “, typical motion patterns are extracted and learned from training data sequences and then learned patterns are used for prediction [2], [12]–[14].”
- Sim-to-Real Robot Learning from Pixels with Progressive Nets (2016, background): “Alternative approaches embrace big data ideas for robotics ([17, 10]).”
- RoboNet: Large-Scale Multi-Robot Learning (2019, background): “Large-scale, self-supervised robot learning approaches have adopted a similar viewpoint [24, 25, 26, 9, 15, 10, 27, 7].”
- A Framework for Data-Driven Robotics (2019, background): “RL: RL has a long history in robotics [14]–[20] but applying RL to robots inherits all the general difficulties of applying RL to the real world [21].”
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control (2018, background): “A number of recent works have studied self-supervised robotic learning, where large-scale unattended data collection is used to learn individual skills such as grasping [22], [23], [24], [25], push-grasp synergies [26], or obstacle avoidance [27], [28].”
- Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, background): “While there have been recent attempts on learning by collecting large scale data directly on real robots [1, 2, 3, 4], such an approach still remains challenging as collecting real world data is prohibitively laborious and expensive.”
- Sim-to-Real: Learning Agile Locomotion For Quadruped Robots (2018, methodology): “urcated consequences. Overcoming the reality gap is challenging. An alternative is to learn the task directly on the physical system. While this has been successfully demonstrated in robotic grasping [9], it is challenging to apply this method Fig. 1: The simulated and the real Minitaurs learned to ”
- Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning (2017, background): “Even when tasks are designed so that these resets are easy (e.g. Levine et al. (2016) and Gu et al. (2017)), manual resets are necessary when the robot or environment breaks (e.g. Gandhi et al. (2017)).”
- Learning to Act by Predicting the Future (2016, methodology): “While the use of SL is natural in imitation learning (LeCun et al., 2005; Ross et al., 2013) or in conjunction with model-based RL (Levine & Koltun, 2013), the formulation of sensorimotor learning from raw experience as supervised learning is rare (Levine et al., 2016).”
- Deep visual foresight for planning robot motion (2016, background): “Several works have used deep neural networks to process images and represent policies for robotic control, initially for driving tasks [15], [16], later for robotic soccer [17], and most recently for robotic grasping [18], [19] and manipulation [20].”
- Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, methodology): “Since then, data-driven algorithms have been scaled up in terms of number of datapoints [3] and shown to be useful for other task such as poking [4].”
- Combining self-supervised learning and imitation for vision-based rope manipulation (2017, background): “Given that state of the art deep reinforcement learning algorithms are data hungry, some recents works learned to grasp [19], [20] and push [21] real world objects by collecting large amounts of robot interaction data.”