Reset-free Trial-and-Error Learning for Robot Damage Recovery

By Konstantinos Chatzilygeroudis, Vassilis Vassiliades, Jean-Baptiste Mouret. 2016

111 citations recorded by the source corpus.

Abstract The high probability of hardware failures prevents many advanced robots (e.g., legged robots) from being confidently deployed in real-world situations (e.g., post-disaster rescue). Instead of attempting to diagnose the failures, robots could adapt by trial-and-error in order to be able to complete their tasks. In this situation, damage recovery can be seen as a Reinforcement Learning (RL) problem. However, the best RL algorithms for robotics require the robot and the environment to be reset to an initial state after each episode, that is, the robot is not learning autonomously. In addition, most of the RL methods for robotics do not scale well with complex robots (e.g., walking robots) and either cannot be used at all or take too long to converge to a solution (e.g., hours of learning). In this paper, we introduce a novel learning algorithm called “Reset-free Trial-and-Error” (RTE) that (1) breaks the complexity by pre-generating hundreds of possible behaviors with a dynamics simulator of the intact robot, and (2) allows complex robots to quickly recover from damage while completing their tasks and taking the environment into account. We evaluate our algorithm on a simulated wheeled robot, a simulated six-legged robot, and a real six-legged walking robot that are damaged in several ways (e.g., a missing leg, a shortened leg, faulty motor, etc.) and whose objective is to reach a sequence of targets in an arena. Our experiments show that the robots can recover most of their locomotion abilities in an environment with obstacles, and without any human intervention.

Subfields: Artificial Intelligence, Biomedical Engineering, Software

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Chelsea Finn
3. Pieter Abbeel
4. Jie Tan
5. Kelvin Xu
6. Trevor Darrell
7. Peng Xu
8. Yoshua Bengio

In Biomedical Engineering

1. Chelsea Finn
2. Sergey Levine
3. Jie Tan
4. Zhenyu Tan
5. Peng Xu
6. Sehoon Ha
7. Pieter Abbeel
8. Vikash Kumar

In Software

1. Jie Tan
2. Chelsea Finn
3. Sergey Levine
4. Peng Xu

Citations in the corpus

• Continual Learning of Control Primitives: Skill Discovery via Reset-Games (2020, background): “Learning without access to resets has been studied in prior work with a focus on automation, safety, and learning compound controllers [23, 13, 8].”
• Learning to Walk in the Real World with Minimal Human Effort (2020, methodology): “Note that our approach is different from repertoire-based adaptation [10, 11] that relies on the pre-obtained gait library.”