Learning to Walk in the Real World with Minimal Human Effort
By Sehoon Ha, Peng Xu, Zhenyu Tan, Sergey Levine, Jie Tan. 2020
224 citations recorded by the source corpus.
Reliable and stable locomotion has been one of the most fundamental challenges for legged robots. Deep reinforcement learning (deep RL) has emerged as a promising method for developing such control policies autonomously. In this paper, we develop a system for learning legged locomotion policies with deep RL in the real world with minimal human effort. The key difficulties for on-robot learning systems are automatic data collection and safety. We overcome these two challenges by developing a multi-task learning procedure, an automatic reset controller, and a safety-constrained RL framework. We tested our system on the task of learning to walk on three different terrains: flat ground, a soft mattress, and a doormat with crevices. Our system can automatically and efficiently learn locomotion skills on a Minitaur robot with little human intervention.
Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Chelsea Finn
- Abhishek Gupta
- Pieter Abbeel
- Karol Hausman
- Trevor Darrell
- Yoshua Bengio
- Ruslan Salakhutdinov
In Biomedical Engineering
- Chelsea Finn
- Sergey Levine
- Abhishek Gupta
- Karol Hausman
- Archit Sharma
- Zipeng Fu
- Ziwen Zhuang
- Soeren Schwertfeger
In Computer Vision and Pattern Recognition
- Sergey Levine
- Chelsea Finn
- Luc Van Gool
- Li Fei-Fei
- Trevor Darrell
- Dacheng Tao
- Jitendra Malik
- Bernt Schiele
Citations in the corpus
- Autonomous Reinforcement Learning via Subgoal Curricula (2021, background): “Some recent works leverage multi-task RL to bypass the need for extrinsic reset mechanisms [15, 14].”
- Robot Parkour Learning (2023, background): “Recently, learning-based methods have shown robust performance on walking [8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 12, 21, 22, 23, 24, 25, 26, 27], climbing stairs [20, 28, 29, 30, 31, 32, 33], mimicking animals [34, 35, 36, 37, 38, 39] and legged mobile manipulation [40, 41, 42] by…”
- MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning (2021, methodology): “Different from existing work which applies Lagrangian to SAC directly [13], we additionally equip SAC-Lag with a PID controller to update the multiplier to alleviate the oscillation in the training [49].”