Learning robot in-hand manipulation with tactile features

By Herke Van Hoof, Tucker Hermans, Gerhard Neumann, Jan Peters. 2015

180 citations recorded by the source corpus.

Dexterous manipulation enables repositioning of objects and tools within a robot's hand. When applying dexterous manipulation to unknown objects, exact object models are not available. Instead of relying on models, compliance and tactile feedback can be exploited to adapt to unknown objects. However, compliant hands and tactile sensors add complexity and are themselves difficult to model. Hence, we propose acquiring in-hand manipulation skills through reinforcement learning, which does not require analytic dynamics or kinematics models. In this paper, we show that this approach successfully acquires a tactile manipulation skill using a passively compliant hand. Additionally, we show that the learned tactile skill generalizes to novel objects.

Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Control and Systems Engineering

In Robotics & Control

Citations in the corpus

Data-efficient Deep Reinforcement Learning for Dexterous Manipulation (2017, background): “…have been routinely applied in robotics, in part because they straightforwardly handle continuous and high-dimensional action spaces [3] and applications include manipulation [26, 13, 25, 37, 18, 5, 39, 8], locomotion e.g. [16, 21], and a range of other challenges such as helicopter ﬂight [1].”
Solving Rubik's Cube with a Robot Hand (2019, background): “[114] learn in-hand manipulation for a simple 3-fingered gripper whereas Kumar et al.”
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations (2017, methodology): “Although such methods have been applied to some dexterous manipulation tasks [52], the tasks are comparatively simpler than those illustrated in our work.”
End-to-End Robotic Reinforcement Learning without Reward Engineering (2019, background): “Reinforcement learning has been applied to a wide variety of robotic manipulation tasks, including grasping objects [19], in-hand object manipulation [30, 38, 32, 23], manipulating flu-”
Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost (2018, methodology): “[15] shows the ability to learn in-hand repositioning in the real world, with a reinforcement learning method (NPREPS) and non-parametric control policies.”
Learning dexterous in-hand manipulation (2018, background): “Conversely, due to the difficulty in modeling such complex systems, there has also been work in approaches that only train on a physical robot [16, 67, 29, 30].”
Deep Dynamics Models for Learning Dexterous Manipulation (2019, background): “Model-free reinforcement learning (RL) methods can learn policies that achieve good performance on complex tasks [2, 3, 4]; however, we will show that these state-of-the-art algorithms struggle when a high degree of ﬂexibility is required, such as moving a pencil to follow arbitrary user-speciﬁed…”
COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning (2020, background): “Robotic RL. RL has been applied to a wide variety of robotic manipulation tasks, including grasping objects [11, 12], in-hand object manipulation [13, 14, 15, 16], pouring ﬂuids [17], door opening [18, 19], and manipulating cloth [20, 21].”
On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulation (2018, background): “In [14], an approach is presented, which exploits tactile-based learning to manipulate an object supported by a planar surface with two robotic ﬁngers.”
Stable reinforcement learning with autoencoders for tactile and visual data (2016, methodology): “In this section, we give a brief overview of non-parametric relative entropy policy search (NP-REPS, introduced in [15], [16]).”
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks (2018, background): “Only a few approaches learn entire manipulation policies through reinforcement only given haptic feedback [27, 28, 51, 53].”
A Review of Tactile Information: Perception and Action Through Touch (2020, methodology): “Action-level information may be computed from contact-level [26], [88], [97] and object-level [115], [123], [134], [135] information as well as directly from tactile signals [27], [144]–[147].”
Learning Dexterous In-Hand Manipulation. (2018, background): “Conversely, due to the difficulty in modeling such complex systems, there has also been work in approaches that only train on a physical robot [16, 67, 29, 30].”
Electronic skins and machine learning for intelligent soft robots (2020, background): “Current progress in in-hand manipulation using tactile sensors is primarily limited to rolling circular objects (90).”
Non-parametric Policy Search with Limited Information Loss (2017, unknown): “This paper draws from our earlier conference papers (Peters et al., 2010; van Hoof et al., 2015a,b).”