Champion-level drone racing using deep reinforcement learning

By Elia Kaufmann, L. Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, Davide Scaramuzza. Nature, 2023

875 citations recorded by the source corpus.

An autonomous system is described that combines deep reinforcement learning with onboard sensors collecting data from the physical world, enabling it to fly faster than human world champion drone pilots around a race track. First-person view (FPV) drone racing is a televised sport in which professional competitors pilot high-speed aircraft through a 3D circuit. Each pilot sees the environment from the perspective of their drone by means of video streamed from an onboard camera. Reaching the level of professional pilots with an autonomous drone is challenging because the robot needs to fly at its physical limits while estimating its speed and location in the circuit exclusively from onboard sensors^ 1 . Here we introduce Swift, an autonomous system that can race physical vehicles at the level of the human world champions. The system combines deep reinforcement learning (RL) in simulation with data collected in the physical world. Swift competed against three human champions, including the world champions of two international leagues, in real-world head-to-head races. Swift won several races against each of the human champions and demonstrated the fastest recorded race time. This work represents a milestone for mobile robotics and machine intelligence^ 2 , which may inspire the deployment of hybrid learning-based solutions in other physical systems.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7fac180a463c3969921381b2ed5f077163f603ff

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Reaching the limit in autonomous racing: Optimal control versus reinforcement learning (2023, background): “…are at the core of every real-world robot and are deployed in an ever-increasing number of applications, such as self-driving cars [1, 2], search and rescue [3], flying cars [4], autonomous vehicle racing [5, 6], navigation of stratospheric balloons [7], and extraterrestrial exploration [8, 9].”
Loss of plasticity in deep continual learning (2024, neutral): “Machine learning and artificial intelligence have made remarkable progress in the past decade, with landmark successes in natural-language processing 2,3 , biology 4 , game playing 5–8 and robotics 9,10 .”