Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach

By Douglas Morrison, Peter Corke, Jürgen Leitner. 2018

649 citations recorded by the source corpus.

This paper presents a real-time, object-independent grasp synthesis method which can be used for closed-loop grasping. Our proposed Generative Grasping Convolutional Neural Network (GG-CNN) predicts the quality and pose of grasps at every pixel. This one-to-one mapping from a depth image overcomes limitations of current deep-learning grasping techniques by avoiding discrete sampling of grasp candidates and long computation times. Additionally, our GG-CNN is orders of magnitude smaller while detecting stable grasps with equivalent performance to current state-of-the-art techniques. The light-weight and single-pass generative nature of our GG-CNN allows for closed-loop control at up to 50Hz, enabling accurate grasping in non-static environments where objects move and in the presence of robot control inaccuracies. In our real-world tests, we achieve an 83% grasp success rate on a set of previously unseen objects with adversarial geometry and 88% on a set of household objects that are moved during the grasp attempt. We also achieve 81% accuracy when grasping in dynamic clutter.

Subfields: Biomedical Engineering, Control and Systems Engineering, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Control and Systems Engineering

In Human-Computer Interaction

Citations in the corpus

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018, neutral): “Finally, in contrast to many current grasping systems that utilize depth sensing [7, 27] or wrist-mounted cameras [23, 27], our method operates on raw monocular RGB observations from an over-the-shoulder camera, and the performance of our method indicates that effective learning can achieve excellen”
Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, background): “Spatial equivariance has previously been shown to improve learning efficiency for vision-based picking [29, 30, 31, 32, 34].”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, methodology): “They utilized the real-time GG-CNN Morrison et al.”
“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer (2019, methodology): “Higher-level methods, on the other hand, assume a simple model for robotic control and focus on bounding box or pose detection for downstream grasp planning [6], [7], [8], [9], [10], [11], [1].”
Learning robust, real-time, reactive robotic grasping (2019, methodology): “To determine the best network configuration, we compare relative performance between our trained networks by evaluating each on detecting ground-truth grasps on a 20% evaluation dataset containing 1,710 augmented images from the Cornell Grasping Dataset.”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “Finally, in contrast to many current grasping systems that utilize depth sensing (Mahler et al. 2018; Morrison et al. 2018) or wrist-mounted cameras (Viereck et al. 2017; Morrison et al. 2018), QT-Opt operates on raw monocular RGB observations from an over-the-shoulder camera that doesn’t need to…”
Learning ambidextrous robot grasping policies (2019, background): “Another way to increase rate is to use feedback policies that actively regrasp dropped objects based on visual servoing (10, 28, 42), force sensing (43–45), or tactile sensing (46–49).”
Grasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost Demonstrations (2019, background): “clutter experiments of [17], [18] (right).”
Spatial Action Maps for Mobile Manipulation (2020, background): “…image segmentation [14] has been shown to beneﬁt from pixel-aligned input and output representations, while end-to-end robotic manipulation [21, 45, 44, 43] is signiﬁcantly more sample efﬁcient when using FCNs to predict state-action values for a dense sampling of grasps aligned with the…”
AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains (2022, methodology): “A frequently used heuristic baseline [23] that keeps tracking the nearest grasp poses across frames is also implemented and compared with our temporal association module.”
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, methodology): “For rectangle based method, we adopt two methods [27, 7] with open implementations.”