Free gait generation with reinforcement learning for a six-legged robot

By Mustafa Suphi Erden, K. Leblebicioğlu. 2008

93 citations recorded by the source corpus.

In this paper the problem of free gait generation and adaptability with reinforcement learning are addressed for a six-legged robot. Using the developed free gait generation algorithm the robot maintains to generate stable gaits according to the commanded velocity. The reinforcement learning scheme incorporated into the free gait generation makes the robot choose more stable states and develop a continuous walking pattern with a larger average stability margin. While walking in normal conditions with no external effects causing unstability, the robot is guaranteed to have stable walk, and the reinforcement learning only improves the stability. The adaptability of the learning scheme is tested also for the abnormal case of deficiency in one of the rear-legs. The robot gets a negative reinforcement when it falls, and a positive reinforcement when a stable transition is achieved. In this way the robot learns to achieve a continuous pattern of stable walk with five legs. The developed free gait generation with reinforcement learning is applied in real-time on the actual robot both for normal walking with different speeds and learning of five-legged walking in the abnormal case.

Subfields: Biomedical Engineering, Mechanical Engineering, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Jeff Clune
2. Jean-Baptiste Mouret
3. Jens Kober
4. J. Bagnell
5. Jan Peters
6. Sergey Levine
7. Kenneth O. Stanley
8. Gerhard Neumann

In Mechanical Engineering

1. Jean-Baptiste Mouret
2. Jan Peters
3. Jens Kober
4. J. Bagnell
5. Antoine Cully
6. Jeff Clune

In Molecular Biology

1. Jens Kober
2. Jan Peters
3. Jean-Baptiste Mouret
4. J. Bagnell
5. Antoine Cully
6. Jeff Clune
7. Christina Heinze-Deml
8. Nicolai Meinshausen

Citations in the corpus

• Reinforcement learning in robotics: A survey (2013, methodology): “For example, a vision-based mobile robot docking task can be learned faster with such a basic behavior than using Q-learning alone, as demonstrated in (Martínez-Marín and Duckett, 2005) Providing hand-coded, stable initial gaits can significantly help in learning robot locomotion, as shown on a six-”
• Robots that can adapt like animals (2014, background): “The results also indicate that Intelligent Trial and Error significantly outperforms previous algorithms for both damage recovery 63,12,64,65,66 and gait learning 67,62,68,48,69,32,61,42,17,49 , and can therefore be considered the state of the art.”