Learning Agile Robotic Locomotion Skills by Imitating Animals

By Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Edward Lee, Jie Tan, Sergey Levine. 2020

669 citations recorded by the source corpus.

Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. While manually-designed controllers have been able to emulate many complex behaviors, building such controllers involves a time-consuming and difficult development process, often requiring substantial expertise of the nuances of each skill. Reinforcement learning provides an appealing alternative for automating the manual effort involved in the development of controllers. However, designing learning objectives that elicit the desired behaviors from an agent can also require a great deal of skill-specific expertise. In this work, we present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals. We show that by leveraging reference motion data, a single learning-based approach is able to automatically synthesize controllers for a diverse repertoire behaviors for legged robots. By incorporating sample efficient domain adaptation techniques into the training process, our system is able to learn adaptive policies in simulation that can then be quickly adapted for real-world deployment. To demonstrate the effectiveness of our system, we train an 18-DoF quadruped robot to perform a variety of agile behaviors ranging from different locomotion gaits to dynamic hops and turns.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Learning quadrupedal locomotion over challenging terrain (2020, positive): “Other work also developed RL techniques for legged locomotion, but likewise focused largely on flat or moderately textured surfaces in laboratory settings [13, 14, 16– 19].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “Robot learning has emerged as a strong paradigm for automatically acquiring complex and skilled behaviors such as locomotion (Yang et al., 2019; Peng et al., 2020), navigation (Mirowski et al., 2017; Kahn et al., 2020), and manipulation (Gu et al., 2017; et al, 2019).”
Legged Locomotion in Challenging Terrains using Egocentric Vision (2022, background): “Alternatively, a policy learned in simulation can be adapted at 239 test-time to work well in real environments [39, 40, 41, 42, 43, 44, 45, 46, 3, 47, 48, 49].”
From Motor Control to Team Play in Simulated Humanoid Football (2021, background): “Yet, recent partial successes transferring results from simulation to real robots (24, 26, 86, 87) suggest that learning based approaches in simulation may, in the future, play a larger role in the control of real world robots.”
Learning robust perceptive locomotion for quadrupedal robots in the wild (2022, methodology): “[35] used imitation learning to transfer animal motion to a legged robot.”
Open-TeleVision: Teleoperation with Immersive Active Visual Feedback (2024, positive): “Besides learning for manipulation, motion imitation for physical characters and real robots have also been widely studied in [75, 76, 40, 77, 78, 79, 80, 81, 82, 80, 83, 84, 85, 86, 87, 88].”
AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning (2021, background): “Moreover, a set of approaches focus on sim2real transfer by adapting the parameters (Yu et al., 2017; Peng et al., 2020).”
Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning (2021, background): “It has also shown impressive results in robotic settings, including legged locomotion [4] and dexterous manipulation [5].”
RMA: Rapid Motor Adaptation for Legged Robots (2021, methodology): “6) Advantage Weighted Regression for Domain Adaptation (AWR) [40]: Optimize ẑt offline using AWR by using real-world rollouts of the policy in the testing environment.”
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos (2021, background): “Another line of work in imitation learning is to train policies to follow the expert demonstrations [43, 52, 49, 71, 68, 72, 74, 21, 51, 73].”
Robot Parkour Learning (2023, background): “In contrast, learning by directly imitating animals’ motion capture data can circumvent tedious reward design and tuning [34, 43], but the lack of egocentric vision data and diverse animal MoCap skills prevents the robots from learning diverse agile skills and autonomously selecting skills by…”
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training (2021, background): “…application area in which HiL methods could have signiﬁcant real-world impact is robotic manipulation, since learning often requires extensive engineering in the real world (Yahya et al., 2017; Schenck & Fox, 2017; Kormushev et al., 2010; Rusu et al., 2017; Akkaya et al., 2019; Peng et al., 2020).”
Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots (2021, methodology): “Randomizing the system properties in the source domain in order to cover the uncertainty in the target domain has allowed for solutions that use low-fidelity simulations for learning-based methods [23], [28], [31]–[35].”