Learning quadrupedal locomotion over challenging terrain

By Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, Marco Hutter. 2020

1,689 citations recorded by the source corpus.

A learning-based locomotion controller enables a quadrupedal ANYmal robot to traverse challenging natural environments. Legged locomotion can extend the operational domain of robots to some of the most challenging environments on Earth. However, conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. These designs have increased in complexity but fallen short of the generality and robustness of animal locomotion. Here, we present a robust controller for blind quadrupedal locomotion in challenging natural environments. Our approach incorporates proprioceptive feedback in locomotion control and demonstrates zero-shot generalization from simulation to natural environments. The controller is trained by reinforcement learning in simulation. The controller is driven by a neural network policy that acts on a stream of proprioceptive signals. The controller retains its robustness under conditions that were never encountered during training: deformable terrains such as mud and snow, dynamic footholds such as rubble, and overground impediments such as thick vegetation and gushing water. The presented work indicates that robust locomotion in natural environments can be achieved by training in simple domains.

Subfields: Aerospace Engineering, Biomedical Engineering, Condensed Matter Physics, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Biomedical Engineering

In Condensed Matter Physics

In Reinforcement Learning

Citations in the corpus

Augmented Language Models: a Survey (2023, methodology): “…with an environment, and it has been used for applications such as playing games (Mnih et al., 2015; Silver et al., 2016; Vinyals et al., 2019; Team et al., 2021; Bakhtin et al., 2022) or controlling robots (Gu et al., 2017; Kalashnikov et al., 2018; Akkaya et al., 2019; Lee et al., 2020).”
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation (2021, background): “Thus, practitioners should take care to engineer the robot observation space and exclude possibly irrelevant information – information-hiding can be a powerful paradigm for training proﬁcient robots [44].”
Meta-Adaptive Nonlinear Control: Theory and Algorithms (2021, background): “Examples abound in robotics, such as a drone flying in different wind conditions [1], a manipulator throwing varying objects [2], or a quadruped walking over changing terrains [3].”
From Motor Control to Team Play in Simulated Humanoid Football (2021, background): “Yet, recent partial successes transferring results from simulation to real robots (24, 26, 86, 87) suggest that learning based approaches in simulation may, in the future, play a larger role in the control of real world robots.”
Extreme Parkour with Legged Robots (2023, neutral): “…• Our results set a new landmark in learning-driven park-our with high jumps that are 2x the height of the robot, long jumps that are 2x the length of the robot, walking on front two legs (handstand), and jumping over titled ramps directly from a single front-facing egocentric depth camera (see…”
Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots (2021, background): “However, most previous RL-based work are deployed on either multi-legged systems [23], [25], [26] or on lowdimensional bipedal robots [27], [28], where learned motions are typically quasi-static.”
Robot Parkour Learning (2023, methodology): “We follow the insights from [10, 8, 35] that use fractal noises to generate terrains, which enforces the foot contact clearance.”
DayDreamer: World Models for Physical Robot Learning (2022, methodology): “4 Related Work Existing work on robot learning commonly leverages large amounts of simulated experience before deploying to the real world (Rusu et al., 2016; Peng et al., 2018; OpenAI et al., 2018; Lee et al., 2020; Irpan et al., 2020; Kumar et al., 2021; Siekmann et al., 2021; Escontrela et al., 2”
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning (2024, background): “Having access to the privileged state can help RL find more optimal solutions, as shown in prior works [50] and our experiments ( Section 4).”
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025, neutral): “…2018; Hafner et al., 2019b,a; Hansen et al., 2022, 2023; Hafner et al., 2023; Schrittwieser et al., 2020; Samsami et al., 2024), as well as real-world locomotion and manipulation tasks (Lee et al., 2020; Nagabandi et al., 2020; Finn et al., 2016; Ebert et al., 2017, 2018; Yen-Chen et al., 2020).”
Real-world humanoid locomotion with reinforcement learning (2023, background): “We hypothesized that the history of observations and actions implicitly encodes the information about the world that a powerful transformer model can use to adapt its behavior dynamically at test time.”
Neural-Fly enables rapid learning for agile flight in strong winds (2022, background): “Another popular deep-learning approach for control in dynamic environments is robust policy learning via domain randomization [42, 43, 44].”
RMA: Rapid Motor Adaptation for Legged Robots (2021, methodology): “The input channel number, output channel number, kernel size, and stride of each layer are [32, 32, 8, 4], [32, 32, 5, 1], [32, 32, 5, 1].”
Perspectives in machine learning for wildlife conservation (2021, background): “ing (146) and robotics (147) in the last decade.”
Legged Locomotion in Challenging Terrains using Egocentric Vision (2022, background): “But 21 given a local direction in which to move, it turns out that both humans [1] and robots [2, 3] can do 22 remarkably well at blind walking.”
A System for General In-Hand Object Re-Orientation (2021, background): “To demonstrate the possibility of transferring learned policies to the real world in the future, we overcome the need for privileged information using the idea of teacher-student training [14, 15].”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “This trick has been successful in a number of settings including robotic manipulation from image pixels (Levine et al. 2016; Pinto et al. 2018), autonomous driving (Chen et al. 2020), and robotic locomotion from a history of proprioceptive sensor measurements (Lee et al. 2020b).”