Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

By Wenshuai Zhao, Jorge Peña Queralta, Tomi Westerlund. IEEE Symposium Series on Computational Intelligence, 2020

1,062 citations recorded by the source corpus.

Deep} reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-toreal gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.

Subfields: Artificial Intelligence, Control and Systems Engineering, Mechanical Engineering

View Paper: https://www.semanticscholar.org/paper/5a1b92aa50797a7c1e99b8840ff01aad66038596

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Yilun Du
3. Dale Schuurmans
4. Jonathan Tompson
5. Pieter Abbeel
6. Mengjiao Yang
7. Jiayu Zhou
8. Chelsea Finn

In Control and Systems Engineering

1. Jonathan Tompson
2. Dale Schuurmans
3. Yilun Du
4. Jiayu Zhou
5. Kaixiang Lin
6. Anil K. Jain
7. Zhuangdi Zhu
8. Mengjiao Yang

In Mechanical Engineering

1. Yilun Du
2. Pieter Abbeel
3. Dale Schuurmans
4. Jonathan Tompson

Citations in the corpus

• Learning Interactive Real-World Simulators (2023, methodology): “• We formulate the action-in-video-out framework as an observation prediction model conditioned on finite history and parametrized by a video diffusion model.”
• Transfer Learning in Deep Reinforcement Learning: A Survey (2020, background): “For instance, [70] consolidated sim-to-real TL methods.”