Learning Hand-Eye Coordination for Robotic Grasping with Large-Scale Data Collection

By Sergey Levine, Peter Pastor, Alex Krizhevsky, Deirdre Quillen. 2016

241 citations recorded by the source corpus.

Subfields: Biomedical Engineering, Control and Systems Engineering, Mechanical Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Control and Systems Engineering

In Mechanical Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018, methodology): “We model the Q-function as a large deep neural network whose architecture, shown in Figure 13, is inspired by the one presented in [25].”
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection (2016, methodology): “This journal article extends our earlier conference paper (Levine et al., 2016) with the addition of a second experimental evaluation on a new robotic platform (the Kuka IIWA) and evaluation of transfer learning with combined data from two different robots.”
IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data (2019, methodology): “Large-Scale Data Collection in Robotics : Self-Supervised Learning has been employed to collect and learn from large amounts of data for tasks such as grasping in both simulated [13, 18, 25] and physical settings [17, 22, 29].”
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation (2018, methodology): “Recent research has successfully employed Reinforcement Learning (RL), Self-Supervised Learning (SSL), and Imitation Learning (IL) for short-horizon skill learning, such as pushing and grasping objects [4, 5, 6].”
Multi-task Deep Reinforcement Learning with PopArt (2018, background): “A limitation of our approach is that it can be complicated to implement parallel learning outside of simulation, but recent work on parallel training of robots (Levine et al. 2016) suggests that this is not necessarily an insurmountable obstacle if sufﬁcient resources are available.”
Zero-Shot Visual Imitation (2018, background): “, 2009; Pomerleau, 1989), which directly supervises the mapping of states to actions, and inverse reinforcement learning (Abbeel & Ng, 2004; Ho & Ermon, 2016; Levine et al., 2016; Ng & Russell, 2000; Ziebart et al., 2008), which recovers a reward function that makes the demonstration optimal (or nea”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “Conventional robot policies based on reinforcement learning have largely focused on addressing a limited set of tasks within controlled environments, such as item grasping [6].”
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning (2018, background): “Recently, using a single differentiable model to map from inputs to outputs across multiple sub-problems has been applied to learning robotic manipulation and control skills [26, 27] and visual navigation in simulated environments [43].”
A Survey of Deep Learning Applications to Autonomous Vehicle Control (2019, background): “However, neural networks have shown great potential for optimising nonlinear, high-dimensional control systems [40], [41], [101]–[106].”
Symbiotic human-robot collaborative assembly (2019, background): “Levine et al. [123] demonstrated the high potential of deep learning algorithms for automated flexible robotic grasping of different objects in undefined poses.”
On Inductive Biases in Deep Reinforcement Learning (2019, background): “1) The use of parallel environments is not always feasible in practice, especially in real world applications (although recent work on robot farms Levine et al. (2016) shows that it might still be a valid approach when sufﬁcient resources are available).”
Unsupervised Learning of Sensorimotor Affordances by Stochastic Future Prediction (2018, background): “Recent breakthroughs in reinforcement learning (RL) suggest that end-to-end training can be used to learn mappings between sensory input and actions [28, 27, 25, 26].”
RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots (2024, background): “One approach is self-supervised learning, where trial-and-error is used to collect data for tasks like grasping and pushing [24, 36, 18, 19, 46, 7].”
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight (2019, methodology): “Representation learning has been studied for executing speciﬁc tasks such as grasping [7, 8, 10], dexterous manipulation [26, 27], or continuous ﬂight [9].”
Learning to Singulate Objects using a Push Proposal Network (2017, unknown): “Our approach is related to robot learning from physical interaction [24], including approaches that learn to predict how rigid objects behave if manipulated [17, 12, 32] and self-supervised learning methods for grasping [25, 20].”
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023, background): “Some robot data collection efforts have employed trial-and-error [12–17] and pre-programmed demonstrators in simulation [18–22], but it can be difficult to scale these approaches to more complex tasks.”
Learning robust, real-time, reactive robotic grasping (2019, methodology): “…and faster to compute than the CNNs used for grasp candidate classification in other works that contain hundreds of thousands (He et al., 2016; Levine et al., 2016) or millions (Johns et al., 2016; Mahler et al., 2017; Pinto and Gupta, 2016; Redmon and Angelova, 2015) of parameters, with a…”