Dex-Net 3.0: Computing Robust Robot Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning

By Jeffrey Mahler, Matthew Matl, Xinyu Liu, Albert Li, David V. Gealy, Ken Goldberg. 2017

133 citations recorded by the source corpus.

Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98$\%$, 82$\%$, and 58$\%$ respectively, improving to 81$\%$ in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at this http URL .

Subfields: Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

Citations in the corpus

Solving Rubik's Cube with a Robot Hand (2019, background): “d to dynamics randomization [80, 3, 105, 119, 77] to learn a robust policy that transfers to similar environments but with different dynamics. Domain randomization was also used to plan robust grasps [65, 66, 107] and to transfer learned locomotion [105] and grasping [123] policies for relatively si”
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, methodology): “DexNet [25, 26] took this further by using cloud computing and synthetic object models to learn grasp models.”
Learning dexterous in-hand manipulation (2018, methodology): “Domain randomization was also used to plan robust grasps [38, 39, 63] and to transfer learned locomotion [61] and grasping [69] policies for relatively simple robots.”
Learning Visual Affordances for Robotic Manipulation (2019, methodology): “Positive labels appear in green while negative labels appear in red. . . 53 xv 3.8 Common Dex-Net failure modes for suction (left column) and parallel-jaw grasping (right column).”
Grasping Field: Learning Implicit Representations for Human Grasps (2020, background): “Recent hybrid approaches [27,32] combine analytic models and deep learning tools to synthesize grasps for various end effectors.”
Learning Deep Policies for Robot Bin Picking by Simulating Robust Grasping Sequences (2017, background): “1 suggest that performance could be improved by introducing alternative actions, such as probing or using a suction cup [37], when the model does not have high confidence.”