6-DOF GraspNet: Variational Grasp Generation for Object Manipulation

By Arsalan Mousavian, Clemens Eppner, Dieter Fox. 2019

695 citations recorded by the source corpus.

Generating grasp poses is a crucial component for any robot object manipulation task. In this work, we formulate the problem of grasp generation as sampling a set of grasps using a variational autoencoder and assess and refine the sampled grasps using a grasp evaluator model. Both Grasp Sampler and Grasp Refinement networks take 3D point clouds observed by a depth camera as input. We evaluate our approach in simulation and real-world robot experiments. Our approach achieves 88% success rate on various commonly used objects with diverse appearances, scales, and weights. Our model is trained purely in simulation and works in the real-world without any extra steps.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks (2019, methodology): “Motivated by work in robotics on segmentation-based grasping [36], agents in ALFRED interact with objects visually, specifying a pixelwise interaction mask of the target object.”
Self-supervised 6D Object Pose Estimation for Robot Manipulation (2019, background): “Learning techniques also demonstrate impressive performance in grasping unknown objects [3]–[5].”
Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, neutral): “1) Datasets: Recently, a substantial number of grasping datasets [156], [157], [158], [159], [160] have been generated.”
Human Hands as Probes for Interactive Object Understanding (2021, background): “This can be in the form of learning a) where and how to grasp [9, 20, 25, 26, 28, 33, 34, 37, 41, 49], b) state classifiers [24], c) interaction hotspots [14, 40, 42, 59], d) spatial priors for action sites [44], e) object articulation modes [11,36], f) reward functions [27,29,48,50], g) functional ”
6-DOF Grasping for Target-driven Object Manipulation in Clutter (2019, methodology): “5D (depth, multi-view) [11], [18], [19] and 3D (point cloud) [20]–”
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation (2022, background): “While other works in 6-DoF grasping [30, 31, 32, 33, 34] have used RGB-D and pointcloud input, they have not been applied to sequential tasks or used with language-conditioning.”
Hand-Object Contact Consistency Reasoning for Human Grasps Generation (2021, background): “Unlike predicting robot grasps with parallel jaw grippers [32, 55, 58, 5], predicting human grasps is substantially more difﬁcult because: (i) Human hands have a lot more degrees of freedom, which leads to much more complex contact; (ii) The generated grasp needs to be not only physically plausible…”
GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes (2020, methodology): “[45] introduced GraspNet to predict 6-DoF for object manipulation.”
“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer (2019, positive): “6-DOF GraspNet [21] uses simulated grasp data to generalize to new objects and has been extended to handle reliable grasping of novel objects in clutter [12].”
Grasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost Demonstrations (2019, methodology): “Most methods only infer top-down grasps (4Dof: x, y, z, yaw) in simple tabletop settings [1]–[4], or detect grasps in 6DoF but with open-loop execution [5], [6].”
Unseen Object Instance Segmentation for Robotic Environments (2020, methodology): “Building such a robust object recognition module is valuable for robots interacting with objects, such as picking up unseen objects or learning to use new tools [1], [2], [3].”
Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation (2020, methodology): “However, there does not exist a large scale dataset of real-world images for many objects in tabletop scenes.”
AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains (2022, methodology): “generate 100 grasp poses on each object, ACRONYM [47] generates 2 k grasp poses and [8] generates 34 k grasp poses.”
Learning Dexterous Grasping with Object-Centric Visual Affordances (2020, methodology): “A common protocol estimates the 6-DoF object pose, followed by modelbased grasp planning [51], [30], [23], [48].”
Grasp'D: Differentiable Contact-rich Grasp Synthesis for Multi-fingered Hands (2022, methodology): “Although analytic metrics have been successfully applied to parallel-jaw gripper grasp synthesis (based on grasp wrench space analysis [27, 35, 63], robust grasp wrench space analysis [60, 86], or caging [62, 74]), more recent works [19, 24, 48, 66] have focused on simulation-based synthesis.”
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, background): “To avoid such problem, some recent works explore using simulated environment [26, 9, 48, 28, 4] to anotate grasp poses.”