RMA: Rapid Motor Adaptation for Legged Robots

By Ashish Kumar, Zipeng Fu, Deepak Pathak, Jitendra Malik. Robotics: Science and Systems Conference, 2021

938 citations recorded by the source corpus.

Successful real-world deployment of legged robots would require them to adapt in real-time to unseen scenarios like changing terrains, changing payloads, wear and tear. This paper presents Rapid Motor Adaptation (RMA) algorithm to solve this problem of real-time online adaptation in quadruped robots. RMA consists of two components: a base policy and an adaptation module. The combination of these components enables the robot to adapt to novel situations in fractions of a second. RMA is trained completely in simulation without using any domain knowledge like reference trajectories or predefined foot trajectory generators and is deployed on the A1 robot without any fine-tuning. We train RMA on a varied terrain generator using bioenergetics-inspired rewards and deploy it on a variety of difficult terrains including rocky, slippery, deformable surfaces in environments with grass, long vegetation, concrete, pebbles, stairs, sand, etc. RMA shows state-of-the-art performance across diverse real-world as well as simulation experiments. Video results at https://ashish-kmr.github.io/rma-legged-robots/

Subfields: Biomedical Engineering, Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/1ca5ff6555d9fc634d3858d1fda9b3de2a91b13a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Genetics

In Molecular Biology

Citations in the corpus

ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (2022, background): “40 [66] Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik.”
Human-Timescale Adaptation in an Open-Ended Task Space (2023, background): “Adaptation also plays a critical role in robotics, with agents trained to adapt to varying terrain (Clavera et al., 2019; Kumar et al., 2021) or damaged joints (Cully et al.”
Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior (2022, background): “Climbing over curbs: Previous works demonstrating blind obstacle traversal on a quadruped [1, 2] learned a foot-trapping reflex where the robot first trips, then raises its foot over the obstacle.”
Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments (2023, background): “Methods such as reinforcement learning (RL) have shown incredible success in challenging problems such as quadrupedal locomotion [1], [2], [3] and”
Robot Parkour Learning (2023, methodology): “We follow the insights from [10, 8, 35] that use fractal noises to generate terrains, which enforces the foot contact clearance.”
Open-TeleVision: Teleoperation with Immersive Active Visual Feedback (2024, neutral): “Learning based methods have achieved great success in locomotion control with massive simulation data and Sim2Real transfer [36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48].”
Real-world humanoid locomotion with reinforcement learning (2023, background): “Among these, the recent approaches for learning legged locomotion have employed either memory-based networks like Long Short-Term Memory (LSTM) ( 14,23 ) or trained an explicit estimator to regress environment properties from Temporal Convolutional Network (TCN) features ( 17, 18 ).”
Rapid locomotion via reinforcement learning (2022, methodology): “This so-called implicit system identification approach has been previously developed in a number of works involving object re-orientation with a multi-finger hand [8], self-driving cars [7] and locomotion [23, 24, 27, 29].”
Evaluating Real-World Robot Manipulation Policies in Simulation (2024, neutral): “This technique has proven successful in applications such as locomotion [26, 36], manipulation with complex dynamics [9, 75], and dexterous in-hand manipulation [2, 56, 72].”
Deep Hierarchical Planning from Pixels (2022, background): “Even training a robot to simply walk forward can require specifying ten different reward terms (Kumar et al., 2021), making reward engineering a critical component of such systems.”
RVT: Robotic View Transformer for 3D Object Manipulation (2023, background): “The learning of robotic control policy has been traditionally studied with low-dimensional state observations [9, 10, 11, 12, 13].”
Foundation models in robotics: Applications, challenges, and the future (2023, neutral): “Developing techniques that perform runtime monitoring and OOD detection with statistical guarantees on false positive/negative error rates in a data-efﬁcient manner remains an important research direction.”
Adapt to Environment Sudden Changes by Learning a Context Sensitive Policy (2022, background): “In recent years, meta RL has achieved significant success in the tasks that have various scenarios (Kumar et al. 2021; Zhang, Yu, and Zhou 2018).”
Extreme Parkour with Legged Robots (2023, neutral): “This has motivated the use of learned controllers trained with RL that can adapt to changes in dynamics [11], [30]–[35] and also leverage elevation maps [9], [36]–[39] for perceptive walking.”
Legged Locomotion in Challenging Terrains using Egocentric Vision (2022, methodology): “But given a local direction in which to move, it turns out that both humans [1] and robots [2, 3] can do remarkably well at blind walking.”
HumanPlus: Humanoid Shadowing and Imitation from Humans (2024, methodology): “To tackle the control complexity of humanoids, we follow the recent success in legged robotics using large-scale reinforcement learning in simulation and sim-to-real transfer [41, 51] to train a low-level policy for whole-body control.”