Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

By Zhongyu Li, Xuxin Cheng, Xue Bin Peng, Pieter Abbeel, Sergey Levine, Glen Berseth, Koushil Sreenath. IEEE International Conference on Robotics and Automation, 2021

296 citations recorded by the source corpus.

Developing robust walking controllers for bipedal robots is a challenging endeavor. Traditional model-based locomotion controllers require simplifying assumptions and careful modelling; any small errors can result in unstable control. To address these challenges for bipedal locomotion, we present a model-free reinforcement learning framework for training robust locomotion policies in simulation, which can then be transferred to a real bipedal Cassie robot. To facilitate sim-to-real transfer, domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. The learned policies enable Cassie to perform a set of diverse and dynamic behaviors, while also being more robust than traditional controllers and prior learning-based methods that use residual control. We demonstrate this on versatile walking behaviors such as tracking a target walking velocity, walking height, and turning yaw. (Video1)

Subfields: Artificial Intelligence, Biomedical Engineering

View Paper: https://www.semanticscholar.org/paper/9789dd9408a4a5116b40ab99603d9143b15a26db

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

Citations in the corpus

Robot Parkour Learning (2023, background): “Recently, learning-based methods have shown robust performance on walking [8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 12, 21, 22, 23, 24, 25, 26, 27], climbing stairs [20, 28, 29, 30, 31, 32, 33], mimicking animals [34, 35, 36, 37, 38, 39] and legged mobile manipulation [40, 41, 42] by…”
Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior (2022, background): “[10, 11] generated a library of reference trajectories and trained a goal-conditioned policy to imitate them.”
Open-TeleVision: Teleoperation with Immersive Active Visual Feedback (2024, unknown): “Learning based methods have achieved great success in locomotion control with massive simulation data and Sim2Real transfer [36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48].”
HumanPlus: Humanoid Shadowing and Imitation from Humans (2024, background): “…humanoid robots [13, 32, 37, 54, 65, 89], learning-based meth-ods can achieve robust locomotion performance for humanoids [39, 67, 68, 85, 91, 94, 103] and biped robots [4, 42, 44, 45, 83, 84, 93, 100] due to their training on highly randomized environments in simulation and their ability to adapt.”
Extreme Parkour with Legged Robots (2023, background): “A parkour expert and novice have access to the same set of “sensors”, and in the process of learning parkour, their sensing capabilities are not significantly improved.”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, methodology): “Notably, some of these works deployed their locomotion policies on humanoid robots (56, 60, 62) while others on bipedal robots without upper bodies (55, 58, 59, 27, 61, 63).”
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning (2024, background): “Recently, learning-based methods have shown promising results [7, 8, 9, 10, 11, 12, 13, 3, 18].”