Grasp Pose Detection in Point Clouds

By Andreas ten Pas, Marcus Gualtieri, Kate Saenko, Robert W. Platt. 2017

673 citations recorded by the source corpus.

Recently, a number of grasp detection methods have been proposed that can be used to localize robotic grasp configurations directly from sensor data without estimating object pose. The underlying idea is to treat grasp perception analogously to object detection in computer vision. These methods take as input a noisy and partially occluded RGBD image or point cloud and produce as output pose estimates of viable grasps, without assuming a known CAD model of the object. Although these methods generalize grasp knowledge to new objects well, they have not yet been demonstrated to be reliable enough for wide use. Many grasp detection methods achieve grasp success rates (grasp successes as a fraction of the total number of grasp attempts) between 75% and 95% for novel objects presented in isolation or in light clutter. Not only are these success rates too low for practical grasping applications, but the light clutter scenarios that are evaluated often do not reflect the realities of real-world grasping. This paper proposes a number of innovations that together result in an improvement in grasp detection performance. The specific improvement in performance due to each of our contributions is quantitatively measured either in simulation or on robotic hardware. Ultimately, we report a series of robotic experiments that average a 93% end-to-end grasp success rate for novel objects presented in dense clutter.

Subfields: Aerospace Engineering, Biomedical Engineering, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Biomedical Engineering

In Control and Systems Engineering

Citations in the corpus

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018, background): “However, current grasping systems typically approach the grasping task as the problem of predicting a grasp pose, where the system looks at the scene (typically using a depth camera), chooses the best location at which to grasp, and then executes an open-loop planner to reach that location [5, 6, 7,”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, background): “State-of-the-art grasping systems typically either operate in an open-loop style, where grasping locations are chosen, and then a motion is executed to complete the grasp [69, 41, 37, 60], or in a closed-loop manner, where grasp prediction is continuously run during motion, either explicitly [65], o”
Grasp2Vec: Learning Object Representations from Self-Supervised Grasping (2018, background): “combine object detection with grasping to be able to grasp target objects by providing class labeled training data for each object [17] and using grasp predictions as object proposals.”
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, methodology): “Following [41], we adopt the simplified gripper model as shown in Fig.”
Pick and Place Without Geometric Object Models (2017, methodology): “However, since REACH-GRASP is intended to reach toward grasp configurations, we use grasp pose detection (GPD) [16], [17] to generate the samples.”
6-DOF GraspNet: Variational Grasp Generation for Object Manipulation (2019, methodology): “In contrast, GPD does not generate many different grasps which some-times leads to situations in which no kinematically feasible grasp can be found.”
Learning robust, real-time, reactive robotic grasping (2019, methodology): “Prior work has proposed methods for overcoming this challenge by performing point cloud fusion from multiple viewpoints, and performing grasp pose detection on the fused point cloud (Arruda et al., 2016; Kahn et al., 2015; ten Pas et al., 2017).”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “Prior work typically tackles grasping as the problem of identifying suitable grasp locations (Zeng et al. 2018; Morrison et. al 2018; Mahler et al. 2018; ten Pas et al. 2017), rather than as an explicit control problem.”
PointNetGPD: Detecting Grasp Configurations from Point Sets (2018, methodology): “5D (depth map) input; some grasping models even require complex hand-crafted features [8] before they can process the data, while very few of them will take the 3D geometry information into consideration [9].”
AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains (2022, methodology): “scene and then evaluate their quality [5], [6], [7], [8].”
Deictic Image Maps: An Abstraction For Learning Pose Invariant Manipulation Policies (2018, neutral): “[25], where the authors use detection in the context of a robotics task.”
Learning Dexterous Grasping with Object-Centric Visual Affordances (2020, neutral): “A common protocol estimates the 6-DoF object pose, followed by modelbased grasp planning [51], [30], [23], [48].”
6-DOF Grasping for Target-driven Object Manipulation in Clutter (2019, background): “Those learningbased approaches that plan full grasp poses are either based on hand-crafted features ([32]–[34]) or have non-learned components [11].”
Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods (2018, background): “Deep learning has been recently incorporated into such systems [20], [22], [24], [26], [32].”
Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes (2021, methodology): “To overcome the sampling complexity issue, grasp locations are often sampled using geometric heuristics [10, 29].”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “objects or to grasp handles of cooking utensils without relearning to grasp from scratch [88, 131, 381].”