High precision grasp pose detection in dense clutter

By Marcus Gualtieri, Andreas ten Pas, Kate Saenko, Robert W. Platt. 2016

327 citations recorded by the source corpus.

This paper considers the problem of grasp pose detection in point clouds. We follow a general algorithmic structure that first generates a large set of 6-DOF grasp candidates and then classifies each of them as a good or a bad grasp. Our focus in this paper is on improving the second step by using depth sensor scans from large online datasets to train a convolutional neural network. We propose two new representations of grasp candidates, and we quantify the effect of using prior knowledge of two forms: instance or category knowledge of the object to be grasped, and pretraining the network on simulated depth data obtained from idealized CAD models. Our analysis shows that a more informative grasp candidate representation as well as pretraining and prior knowledge significantly improve grasp detection. We evaluate our approach on a Baxter Research Robot and demonstrate an average grasp success rate of 93% in dense clutter. This is a 20% improvement compared to our prior work.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Human-Computer Interaction

Citations in the corpus

Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning (2018, background): “rience [7], [8] or induced by grasp stability metrics [9], [10].”
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection (2016, methodology): “, 2015) that may be mounted on the wrist to establish a clearer image (Gualtieri et al., 2016), and typically use a geometric path planner to actually plan and execute a grasp (Gualtieri et al.”
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping (2018, background): “dered simulation-toreal world transfer in the context of robotic manipulation. Saxena et al. [11] used rendered objects to learn a visionbased grasping model. Gulatieri et al. and Viereck et al. [4], [12] have considered simulation-to-real world transfer using depth images. Depth images can abstract”
kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation (2019, methodology): “Once we know which aspect of the local geometry to focus on, a high quality grasp can be found by any variety of geometric or learning-based grasping algorithms [7, 11].”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, background): “Other works have considered simulation-to-real world transfer using only depth images [64, 18].”
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes? (2017, unknown): “[21] detected grasping poses from depth data, using convolutional neural network trained on simulated data.”
Helping Robots Learn: A Human-Robot Master-Apprentice Model Using Demonstrations via Virtual Reality Teleoperation (2020, methodology): “Convolutional neural networks [33] have also been used [34], [32], [35], [36], [37], [38], typically predicting feasibility of a set of grasping hypotheses.”
Learning a visuomotor controller for real world robotic grasping using simulated depth images (2017, methodology): “the heuristics outlined in our prior work [2].”
GPLAC: Generalizing Vision-Based Robotic Skills Using Weakly Labeled Images (2017, methodology): “Prior methods have also sought to overcome the robotic data challenge with brute force, pooling large volumes of data from extensive robotic experience [39, 28] or simulation [19, 16, 33].”
Deep learning a grasp function for grasping under gripper pose uncertainty (2016, methodology): “In [32], grasping in clutter was achieved by using a static stability”
Pick and Place Without Geometric Object Models (2017, methodology): “However, since REACH-GRASP is intended to reach toward grasp configurations, we use grasp pose detection (GPD) [16], [17] to generate the samples.”
Robotic grasp detection using deep convolutional neural networks (2016, background): “[29] for removing objects from a dense cluster.”
Grasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost Demonstrations (2019, background): “[18] 7 7 simulation [23, 37, 36, 15, 14] 7 7 real [8, 17, 21, 7] 7 3 simulation [30, 19] 3 7 simulation [20, 13] 3 7 real Ours 3 3 real”
Learning Deep Policies for Robot Bin Picking by Simulating Robust Grasping Sequences (2017, methodology): “8 computed from edge detection in depth images inspired by [10, 9].”
Grasp Pose Detection in Point Clouds (2017, methodology): “ing grasps on a speciﬁc object of interest by combining object and grasp detection. The contributions above incorporate and extend work from two prior conference publications (ten Pas and Platt 2015; Gualtieri et al. 2016). In ten Pas and Platt (2015), we proposed the grasp candidate sampling strate”
Learning Visual Affordances for Robotic Manipulation (2019, background): “Many recent successful approaches to learning grasping policies, maximize affordance metrics learned from experience [88, 125] or induced by grasp stability metrics [36, 67].”
Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching (2017, methodology): “out object identity to efficiently detect grasps [11], [12], [13], [14], [15], [16], [17], [18], [19].”
Deictic Image Maps: An Abstraction For Learning Pose Invariant Manipulation Policies (2018, background): “2017) and (Gualtieri et al. 2016), however the focus here is on more complex manipulation tasks rather than just grasping.”
6-DOF Grasping for Target-driven Object Manipulation in Clutter (2019, background): “structured clutter: Most learning-based grasping approaches for clutter deal with rather small and light objects that are spread in a bin ([13], [14], [27], [28]).”