How to train your robot with deep reinforcement learning: lessons we have learned

By Julian Ibarz, Jie Tan, Chelsea Finn, Mrinal Kalakrishnan, Peter Pastor, Sergey Levine. Int. J. Robotics Res., 2021

684 citations recorded by the source corpus.

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

Subfields: Artificial Intelligence, Control and Systems Engineering, Mechanical Engineering

View Paper: https://www.semanticscholar.org/paper/400811ee31020a3f002551476dac25973e13035e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Mechanical Engineering

Citations in the corpus

Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “…do not explicitly address this topic, e.g., Dulac-Arnold et al. (9) discuss the general challenges of real-world RL not specific to robotics, and Ibarz et al. (10) list open challenges of DRL unique to real-world robotics settings but based on case studies drawn only from their own research.”
Continual Semantic Segmentation with Automatic Memory Sample Selection (2023, background): “Reinforcement learning (RL) has achieved remarkable success in many decision-making tasks like game intelligence [41] and robot control [24, 27].”
Reinforcement Learning Control of a Forestry Crane Manipulator (2021, background): “nt advances in machine learning (ML) in general, and reinforcement learning (RL) in particular, have inspired progress in the development of intelligent systems in the context of robotic manipulation [1,2]. These tasks often require multiple-skill acquisition in high-dimensional and dynamically comp”