Humanoid robots learning to walk faster: from the real world to simulation and back
By Alon Farchy, Samuel Barrett, Patrick MacAlpine, Peter Stone. 2013
97 citations recorded by the source corpus.
Simulation is often used in research and industry as a low cost, high efficiency alternative to real model testing. Simulation has also been used to develop and test powerful learning algorithms. However, parameters learned in simulation often do not translate directly to the application, especially because heavy optimization in simulation has been observed to exploit the inevitable simulator simplifications, thus creating a gap between simulation and application that reduces the utility of learning in simulation. This paper introduces Grounded Simulation Learning (GSL), an iterative optimization framework for speeding up robot learning using an imperfect simulator. In GSL, a behavior is developed on a robot and then repeatedly: 1) the behavior is optimized in simulation; 2) the resulting behavior is tested on the real robot and compared to the expected results from simulation, and 3) the simulator is modified, using a machine-learning approach to come closer in line with reality. This approach is fully implemented and validated on the task of learning to walk using an Aldebaran Nao humanoid robot. Starting from a set of stable, hand-coded walk parameters, four iterations of this three-step optimization loop led to more than a 25% increase in the robot's walking speed.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Harish Chaandar Ravichandar
- Yevgen Chebotar
- Sonia Chernova
- Yoshua Bengio
- Dieter Fox
Citations in the corpus
- Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, methodology): “However, [32] requires a human in the loop to select the best simulation parameters, and [33] needs to fit additional models for the real robot forward dynamics and simulator inverse dynamics.”
- A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “An iterative optimization algorithm [85] Real-world experiments The Aldebaran Nao of Humanoid robots [319]”
- Grounded Action Transformation for Robot Learning in Simulation (2017, methodology): “The first is the open source SimSpark simulator.3 The simulator simulates realistic physics with the Open Dynamics Engine.4 This simulator was also used in the work introducing GSL (Farchy et al. 2013).”
- Recent Advances in Robot Learning from Demonstration (2020, methodology): “LfD approaches have been successfully used to enable bipedal robots to learn to walk (177, 178) and optimize their gaits (179).”