Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards

By Jeffrey Mahler, Florian T. Pokorny, Brian Hou, Melrose Roderick, Michael Laskey, Mathieu Aubry, Kai J. Kohlhoff, Torsten Kröger, James J. Kuffner Jr., Ken Goldberg. 2016

391 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection (2016, methodology): “For example, Dex-Net used a dataset of 10,000 3D models, combined with a learning framework to acquire force closure grasps (Mahler et al., 2016), while the work of Oberlin & Tellex (2015) proposed autonomously collecting object scans using a Baxter robot.”
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, methodology): “DexNet [25, 26] took this further by using cloud computing and synthetic object models to learn grasp models.”
The Curious Robot: Learning Visual Representations via Physical Interactions (2016, background): “ds of such pairs to provide training examples for learning ConvNets. Vision and Deep Learning for Robotics. There has been a recent trend to use deep networks in robotics systems for grasp regression [25,35,36] or learning policies for a variety of tasks [37,38]. For example, [39] show how deep visu”
Supervision via competition: Robot adversaries for learning tasks (2016, background): “Significant progress has been made recently using data-driven solutions for training deep learned models on both corporeal robots [1]–[4] and in simulation [34], [35].”
kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation (2019, methodology): “niﬁcant advances in grasping algorithms that allow robots to reliably pick up a wide range of objects, including potentially unseen objects. Among various approaches for grasping, model-based methods [12,32] typically rely on a pre-built grasp database of common 3D object models labeled with sets of”
Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias (2018, background): “For large-scale data collection both simulation [24, 34] and real-world robots [4, 7] have been used.”
Learning robust, real-time, reactive robotic grasping (2019, result): “While this is a relatively small grasping dataset compared with some other, synthetic data-sets (Mahler et al., 2017, 2016), the data best suits our pix-elwise grasp representation as multiple labeled grasps are provided per image.”
6-DOF Grasping for Target-driven Object Manipulation in Clutter (2019, background): “Although multiple learning-based approaches for dealing with grasping in clutter exist ([11], [13], [14], [27]) we will show in the following that they differ from our approach in multiple aspects.”
ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills (2023, background): “At the core is grasp pose Mahler et al. (2016); Qin et al. (2020); Sundermeyer et al. (2021).”
Dex-Net 3.0: Computing Robust Robot Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning (2017, methodology): “Finally, we sample a point cloud of the scene using rendering and a model of image noise [23].”
Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data (2020, background): “While task agnostic manipulation requires geometric understanding of the objects being manipulated, ﬁne-grained understanding of the object conﬁguration is often unnecessary to effectively grasp or push an object [7, 15, 19, 22–24].”
Learning Grasping Interaction with Geometry-aware 3D Representations (2017, background): “…grasping planning and control often involves 3D modeling of object shape, modeling dynamics of robot hands, and local surface modeling Goldfeder et al. [2009], Leon et al. [2010], Johns et al. [2016], Varley et al. [2016], Li et al. [2016], Vahrenkamp et al. [2016], Mahler et al. [2016, 2017].”
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes? (2017, neutral): “As an alternative, data-driven approaches have sought to predict grasp outcomes from human supervision [15, 16, 7], simulation [17, 18, 19], or autonomous robotic data collection [8, 9], typically using visual or depth observations.”
Shape completion enabled robotic grasping (2016, methodology): “Our work differs from [14][15], both of which require a complete mesh to query a model database and retrieve grasps TABLE I: CNN Architecture”
Grasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost Demonstrations (2019, background): “However, most prior self-supervised grasping systems are often constrained to learning in simulation [15], [17], [20], [24] or structured lab environments [1], [3], [21], [22].”
ContactPose: A Dataset of Grasps with Object Contact and Hand Pose (2020, background): “In simulation [63] and robotic grasping [38, 40], contact is often determined similarly, or through collision detection [33,58].”