On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulation

By Pietro Falco, Abdallah Attawia, Matteo Saveriano, Dongheui Lee. 2018

32 citations recorded by the source corpus.

In this letter, we present an approach for learning in-hand manipulation skills with a low-cost, underactuated prosthetic hand in the presence of irreversible events. Our approach combines reinforcement learning based on visual perception with low-level reactive control based on tactile perception, which aims to avoid slipping. The objective of the reinforcement learning level consists not only in fulfilling the in-hand manipulation goal, but also in minimizing the intervention of the tactile reactive control. This way, the occurrence of object slipping during the learning procedure, which we consider an irreversible event, is significantly reduced. When an irreversible event occurs, the learning process is considered failed. We show the performance in two tasks, which consist in reorienting a cup and a bottle only using the fingers. The experimental results show that the proposed architecture allows reaching the goal in the Cartesian space and reduces significantly the occurrence of object slipping during the learning procedure. Moreover, without the proposed synergy between reactive control and reinforcement learning it was not possible to avoid irreversible events and, therefore, to learn the task.

Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Pieter Abbeel
3. Chelsea Finn
4. Quoc V. Le
5. Marcin Andrychowicz
6. Yoshua Bengio
7. Trevor Darrell
8. Ruslan Salakhutdinov

In Biomedical Engineering

1. Joshua Tobin
2. Marcin Andrychowicz
3. Wojciech Zaremba
4. Peter Welinder
5. Bob McGrew
6. OpenAI

In Control and Systems Engineering

1. Sergey Levine
2. Marcin Andrychowicz
3. OpenAI
4. Bob McGrew
5. Joshua Tobin
6. Glenn Powell
7. Alex Ray
8. Lilian Weng

Citations in the corpus

• Solving Rubik's Cube with a Robot Hand (2019, background): “[31] learn such policies for more complex humanoid hands.”
• Learning dexterous in-hand manipulation (2018, background): “Conversely, owing to the difficulty in modeling such complex systems, there has also been work in approaches that only train on a physical robot (Falco et al., 2018; Kumar et al., 2016a,b; van Hoof et al., 2015).”
• Learning Dexterous In-Hand Manipulation. (2018, background): “Conversely, due to the difficulty in modeling such complex systems, there has also been work in approaches that only train on a physical robot [16, 67, 29, 30].”