Learning to Manipulate Unknown Objects in Clutter by Reinforcement

By Abdeslam Boularias, J. Bagnell, Anthony Stentz. 2015

100 citations recorded by the source corpus.

We present a fully autonomous robotic system for grasping objects in dense clutter. The objects are unknown and have arbitrary shapes. Therefore, we cannot rely on prior models. Instead, the robot learns online, from scratch, to manipulate the objects by trial and error. Grasping objects in clutter is significantly harder than grasping isolated objects, because the robot needs to push and move objects around in order to create sufficient space for the fingers. These pre-grasping actions do not have an immediate utility, and may result in unnecessary delays. The utility of a pre-grasping action can be measured only by looking at the complete chain of consecutive actions and effects. This is a sequential decision-making problem that can be cast in the reinforcement learning framework. We solve this problem by learning the stochastic transitions between the observed states, using nonparametric density estimation. The learned transition function is used only for re-calculating the values of the executed actions in the observed states, with different policies. Values of new state-actions are obtained by regressing the values of the executed actions. The state of the system at a given time is a depth (3D) image of the scene. We use spectral clustering for detecting the different objects in the image. The performance of our system is assessed on a robot with real-world objects.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning (2018, background): “[12], which explores the use of reinforcement learning for training control policies to select among push and grasp proposals represented by hand-crafted features.”
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours (2015, background): “Hence the grasp conﬁguration lies in 3 dimensions, ( x, y ) : position of grasp point on the surface of table and θ : angle of grasp.”
Learning Visual Affordances for Robotic Manipulation (2019, background): “[14], which explores the use of reinforcement learning for training control policies to select among push and grasp proposals represented by hand-crafted features.”
Interactive Perception: Leveraging Action in Perception and Perception in Action (2016, methodology): “This emphasizes the blurred boundary which is traditionally drawn between manipulation and perception.”
Deep learning a grasp function for grasping under gripper pose uncertainty (2016, background): “For determining object grasp poses from images, use of hand-engineered features still performs well in complex cases such as dense clutter [30] or multi-fingered grasping [17].”
Learning to Singulate Objects using a Push Proposal Network (2017, methodology): “[3] learn favourable push actions to improve grasping of objects in clutter in a reinforcement learning setting, but use manual features and do not show the applicability of their method for more than 2 objects.”