Robotic grasp detection using deep convolutional neural networks

By Sulabh Kumra, Christopher Kanan. IEEE/RJS International Conference on Intelligent RObots and Systems, 2016

471 citations recorded by the source corpus.

Deep learning has significantly advanced computer vision and natural language processing. While there have been some successes in robotics using deep learning, it has not been widely adopted. In this paper, we present a novel robotic grasp detection system that predicts the best grasping pose of a parallel-plate robotic gripper for novel objects using the RGB-D image of the scene. The proposed model uses a deep convolutional neural network to extract features from the scene and then uses a shallow convolutional neural network to predict the grasp configuration for the object of interest. Our multi-modal model achieved an accuracy of 89.21% on the standard Cornell Grasp Dataset and runs at real-time speeds. This redefines the state-of-the-art for robotic grasp detection.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/7e35cb2ac9eab668ad28244a1ed6c21689dc7fae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Human-Computer Interaction

Citations in the corpus

Learning robust, real-time, reactive robotic grasping (2019, methodology): “…approaches typically use adapted versions of convolutional neural network (CNN) architectures designed for object recognition (Johns et al., 2016; Kumra and Kanan, 2017; Pinto and Gupta, 2016; Redmon and Angelova, 2015), and in most cases sample and rank thousands of grasp candidates…”
The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints (2018, methodology): “We implemented several models similar to those found in existing work[1], [36], [21], [37].”
“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer (2019, methodology): “Higher-level methods, on the other hand, assume a simple model for robotic control and focus on bounding box or pose detection for downstream grasp planning [6], [7], [8], [9], [10], [11], [1].”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, methodology): “(2018), PointPillars Lang et al. (2019), TANet Liu et al. (2020), HVNet Ye et al.”
Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach (2018, methodology): “However, these approaches typically use adapted versions of Convolutional Neural Network (CNN) architectures designed for object recognition [12, 15, 23, 25], and in most cases sample and rank grasp candidates individually [17, 21, 23], resulting in long computation times in the order of a second…”
Real-World Multiobject, Multigrasp Detection (2018, methodology): “Furthermore, we propose to transform grasp configuration from a regression problem formulated in previous works [20], [21] into a combination of region detection and orientation classification problems (with null hypothesis competition).”
A Joint Network for Grasp Detection Conditioned on Natural Language Commands (2021, background): “While classical approaches are based on mechanical constrains [14], [15], recent studies resorted to CNN to capture the geometric information to either predict quality scores for sampled grasps [16], to directly output the grasp configurations [17]–[20], or to capture collision information [21].”
Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network (2019, methodology): “Instead of the 5 dimensional grasp representation used in [1], [2], [4], we use an improved version of grasp representation similar to the one proposed by Morrison et al.”