The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics

By Sylvain Koos, Jean-Baptiste Mouret, Stéphane Doncieux. 2013

303 citations recorded by the source corpus.

The reality gap, which often makes controllers evolved in simulation inefficient once transferred onto the physical robot, remains a critical issue in evolutionary robotics (ER). We hypothesize that this gap highlights a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: the most efficient solutions in simulation often exploit badly modeled phenomena to achieve high fitness values with unrealistic behaviors. This hypothesis leads to the transferability approach, a multiobjective formulation of ER in which two main objectives are optimized via a Pareto-based multiobjective evolutionary algorithm: 1) the fitness; and 2) the transferability, estimated by a simulation-to-reality (STR) disparity measure. To evaluate this second objective, a surrogate model of the exact STR disparity is built during the optimization. This transferability approach has been compared to two reality-based optimization methods, a noise-based approach inspired from Jakobi's minimal simulation methodology and a local search approach. It has been validated on two robotic applications: 1) a navigation task with an e-puck robot; and 2) a walking task with a 8-DOF quadrupedal robot. For both experimental setups, our approach successfully finds efficient and well-transferable controllers only with about ten experiments on the physical robot.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Robotics & Control

Citations in the corpus

Domain randomization for transferring deep neural networks from simulation to the real world (2017, neutral): “[22].”
Go-Explore: a New Approach for Hard-Exploration Problems (2019, background): “in robotics), one can then use any of the many available techniques for transferring the robust policy from simulation to the real world [59, 60, 81].”
Off-Policy Evaluation via Off-Policy Classification (2019, methodology): “While to our knowledge no prior work has proposed a classiﬁcation-based OPE approach, several prior works have used supervised classiﬁers to predict transfer performance from a few runs in the test environment [17, 18].”
A scalable pipeline for designing reconfigurable organisms (2020, background): “For instance, a filter could be added which preemptively steers the evolutionary algorithm away from portions of the design space known to contain designs that cannot be realized physically (25).”
Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation (2020, unknown): “This problem has longer been appreciated, and is particularly salient, in the field of robotics, where there is an inevitable reality gap [25, 11] between the simulated environments used for training and deployment in the real-world.”
A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research (2019, background): “However, for a simulation to be the most identical to reality, perfect models are needed to minimize the reality gap [90,91].”
Quality and Diversity Optimization: A Unifying Modular Framework (2017, background): “This can happen when the optimization process takes place in simulation, and the obtained result does not transfer well to reality (a phenomenon called the reality gap [11]).”