Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learning

By Jemin Hwangbo, Marco Hutter. 2019

112 citations recorded by the source corpus.

The ability to recover from a fall is an essential feature for a legged robot to navigate in challenging environments robustly. Until today, there has been very little progress on this topic. Current solutions mostly build upon (heuristically) predefined trajectories, resulting in unnatural behaviors and requiring considerable effort in engineering system-specific components. In this paper, we present an approach based on model-free Deep Reinforcement Learning (RL) to control recovery maneuvers of quadrupedal robots using a hierarchical behavior-based controller. The controller consists of four neural network policies including three behaviors and one behavior selector to coordinate them. Each of them is trained individually in simulation and deployed directly on a real system. We experimentally validate our approach on the quadrupedal robot ANYmal, which is a dog-sized quadrupedal system with 12 degrees of freedom. With our method, ANYmal manifests dynamic and reactive recovery behaviors to recover from an arbitrary fall configuration within less than 5 seconds. We tested the recovery maneuver more than 100 times, and the success rate was higher than 97 %.

Subfields: Artificial Intelligence, Biomedical Engineering, Cell Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Cell Biology

Citations in the corpus

Learning quadrupedal locomotion over challenging terrain (2020, positive): “The main reason for using analytic IK is to maximize computational efficiency and to reuse existing position control actuator models [6, 15] for the sim-to-real transfer.”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “The interface between the DRL policy and the model-based controller could be defined at various levels, such as joint positions (31, 32, 33), leg poses (28), gait parameters (34, 35), or temporally-extended macro actions (36).”
Language to Rewards for Robotic Skill Synthesis (2023, background): “However, in many previous works in reinforcement learning (RL) or model predictive control (MPC), manual reward design requires extensive domain expertise [17, 18, 19].”
Catch & Carry (2019, background): “Narrow behaviors may be appropriate for certain animation settings, or even medical applications in which musculo-tendon models can help characterize the impact of surgical interventions [Holzbaur et al. 2005; Lee et al. 2019].”
Learning robust perceptive locomotion for quadrupedal robots in the wild (2022, background): “[3, 41] realized quadrupedal locomotion and recovery on flat ground with a physical robot by using learned actuator dynamics to facilitate simulation-toreality (sim-to-real) transfer.”