Pick and Place Without Geometric Object Models

By Marcus Gualtieri, Andreas ten Pas, Robert W. Platt. 2017

58 citations recorded by the source corpus.

We propose a novel formulation of robotic pick and place as a deep reinforcement learning (RL) problem. Whereas most deep RL approaches to robotic manipulation frame the problem in terms of low level states and actions, we propose a more abstract formulation. In this formulation, actions are target reach poses for the hand and states are a history of such reaches. We show this approach can solve a challenging class of pick-place and regrasping problems where the exact geometry of the objects to be handled is unknown. The only information our method requires is: 1) the sensor perception available to the robot at test time; 2) prior knowledge of the general class of objects for which the system was trained. We evaluate our method using objects belonging to two different categories, mugs and bottles, both in simulation and on real hardware. Results show a major improvement relative to a shape primitives baseline.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation (2019, methodology): “rmulate how to specify and solve the task of manipulating objects into speciﬁc conﬁgurations. As a different route to address category-level pick and place, without an explicit object representation, [6] trains end-to-end policies in simulation to generalize across the object category. It is unclear”
Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation (2021, background): “Direct learning of pick-and-place policies meanwhile requires large amounts of data from demonstrations [2, 14, 40].”
Visual Room Rearrangement (2021, background): “…complete observability of the state from perfect visual perception [11, 27], a planar surface as the environment [30, 42], a static robot [15, 32], same environment for evaluation of generalization [39, 26], or a limited set of object categories or limited variation within the categories [10, 19].”
Self-Supervised Learning for Precise Pick-and-Place Without Object Model (2020, background): “[18] learned pick-and-place of 6-degrees of freedom (DoF) in simulation and were able to transfer the results to real-world cluttered scenes.”