Real-world humanoid locomotion with reinforcement learning

By Ilija Radosavovic, Tete Xiao, Bike Zhang, T. Darrell, Jitendra Malik, Koushil Sreenath. Science Robotics, 2023

394 citations recorded by the source corpus.

Humanoid robots that can autonomously operate in diverse environments have the potential to help address labor shortages in factories, assist elderly at home, and colonize new planets. Although classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesized that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in context, without updating its weights. We trained our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deployed it to the real-world zero-shot. Our controller could walk over various outdoor terrains, was robust to external disturbances, and could adapt in context. A reinforcement learning–based controller enables real-world humanoid locomotion. Editor’s summary The ability of robots to navigate adaptively and robustly in varying terrain increases their chances of success when deployed in the real world. However, stable locomotion of full-size bipedal humanoid robots creates a challenge from a controls perspective. Radosavovic et al. developed a reinforcement learning approach for controlling locomotion of a humanoid robot, Digit. They trained their model in simulation and subsequently deployed it into the real-world zero-shot and showed the potential for robust locomotion on various indoor and outdoor environments. The robot could exhibit natural and adaptive walking behaviors, including an emergent arm-swing motion, and adapt to external perturbations. —Amos Matsiko

Subfields: Artificial Intelligence, Biomedical Engineering

View Paper: https://www.semanticscholar.org/paper/a8f842ed0dec8d81e3530ad26fc05512c8958198

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

Citations in the corpus

Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, methodology): “Notably, some of these works deployed their locomotion policies on humanoid robots (56, 60, 62) while others on bipedal robots without upper bodies (55, 58, 59, 27, 61, 63).”
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning (2024, background): “Recently, learning-based methods have shown promising results [7, 8, 9, 10, 11, 12, 13, 3, 18].”