Learning dexterous manipulation for a soft robotic hand from human demonstrations

By Abhishek Gupta, Clemens Eppner, Sergey Levine, Pieter Abbeel. 2016

197 citations recorded by the source corpus.

Dexterous multi-fingered hands can accomplish fine manipulation behaviors that are infeasible with simple robotic grippers. However, sophisticated multi-fingered hands are often expensive and fragile. Low-cost soft hands offer an appealing alternative to more conventional devices, but present considerable challenges in sensing and actuation, making them difficult to apply to more complex manipulation tasks. In this paper, we describe an approach to learning from demonstration that can be used to train soft robotic hands to perform dexterous manipulation tasks. Our method uses object-centric demonstrations, where a human demonstrates the desired motion of manipulated objects with their own hands, and the robot autonomously learns to imitate these demonstrations using reinforcement learning. We propose a novel algorithm that allows us to blend and select a subset of the most feasible demonstrations, which we use with an extension of the guided policy search framework that learns generalizable neural network policies. We demonstrate our approach on the RBO Hand 2, with learned motor skills for turning a valve, manipulating an abacus, and grasping.

Subfields: Aerospace Engineering, Biomedical Engineering, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Biomedical Engineering

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Learning dexterous manipulation skills with multi-ﬁngered hands, for which model-based (Kumar et al., 2016; Gupta et al., 2016) and model-free (Popov et al., 2017) DRL algorithms have been proposed and demonstrated in real robotic experiments, is quite challenging.”
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations (2017, background): “Many of the past successes in dexterous manipulation have focused either on designing hands that mechanically simplify the control problem [8], [14], at the”
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation (2018, methodology): “Prior works show that data-driven IL methods improve both the sample efﬁciency of policy learning [16, 17, 18] and the performance of the trained policy [19, 20, 21].”
Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost (2018, background): “Techniques such as [13], [14] get around this problem”
Third-Person Imitation Learning (2017, background): “Such a mapping is often difﬁcult to obtain, and it typically relies on providing feature representations that captures the invariance between both environments Carpenter et al. (2002); Shon et al. (2005); Calinon et al. (2007); Nehaniv (2007); Gioioso et al. (2013); Gupta et al. (2016).”
Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model (2016, background): “Flexible or inflatable bodies [35] [13], area contact [12], interaction with fluids [34] [31] are just a few of such examples.”
Learning Dexterous Manipulation Policies from Experience and Imitation (2016, neutral): “This approach resembles the work of (Gupta, Eppner, Levine and Abbeel 2016), which used an EM-style algorithm to associate demonstrations with initial states in a reinforcement learning scenario.”
“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer (2019, background): “Object-centric skill learning can be effective and generalize well, e.g. , [13]–[16] focus on block stacking by classifying simulated stacks as stable orlikelytofall.”
Learning Dexterous Grasping with Object-Centric Visual Affordances (2020, background): “, Rubik’s cube [1]) or by incorporating explicit human demonstrations [46], [12], [39], [15], [57], [47], [38].”
A System for General In-Hand Object Re-Orientation (2021, methodology): “To reduce the sample complexity of model-free learning, [9, 13, 8, 41, 6] combined imitation learning with RL to learn to reorient a pen, open a door, assemble LEGO blocks, etc.”
The Ingredients of Real-World Robotic Reinforcement Learning (2020, background): “ environment more representative of the real world, free of instrumentation. While all three prior works utilize instrumentation for resets, state estimation, and reward, the motion capture system of Gupta et al. (2016), sensor attached to the door in Zhu et al. (2019), and auxiliary robot which pic”
Manipulators and Manipulation in high dimensional spaces (2016, methodology): “This approach resembles the work of [35], which used an EM-”
Recent Advances in Robot Learning from Demonstration (2020, methodology): “For instance, the dynamical movement primitives (99) algorithm has been utilized in a variety of platforms, including manipulators (152), robotic hands (67, 199), humanoids (200), and aerial vehicles (201).”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “send a control signal to its actuators to perform actions in the physical world. These signals may, for example, dene the desired pressure for a hydraulic or pneumatic actuator (Bischo et al., 2014; Gupta et al., 2016), the heating for a shape memory alloy actuator (AbuZaiter et al., 2016; Zimmer et”
On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulation (2018, methodology): “ithtworoboticﬁngers.A ReFlex robot hand is used, which is compliant and underactuated. A method to learn ﬁnger-level manipulation skills with a low-cost hand from human demonstrations is presented in [15]. Even though this work does not show case studies concerning in-hand manipulation, the programm”