Deep Drone Racing: Learning Agile Flight in Dynamic Environments

By Elia Kaufmann, Antonio Loquercio, René Ranftl, Alexey Dosovitskiy, Vladlen Koltun, Davide Scaramuzza. 2018

154 citations recorded by the source corpus.

Autonomous agile flight brings up fundamental challenges in robotics, such as coping with unreliable state estimation, reacting optimally to dynamically changing environments, and coupling perception and action in real time under severe resource constraints. In this paper, we consider these challenges in the context of autonomous, vision-based drone racing in dynamic environments. Our approach combines a convolutional neural network (CNN) with a state-of-the-art path-planning and control system. The CNN directly maps raw images into a robust representation in the form of a waypoint and desired speed. This information is then used by the planner to generate a short, minimum-jerk trajectory segment and corresponding motor commands to reach the desired goal. We demonstrate our method in autonomous agile flight scenarios, in which a vision-based quadrotor traverses drone-racing tracks with possibly moving gates. Our method does not require any explicit map of the environment and runs fully onboard. We extensively test the precision and robustness of the approach in simulation and in the physical world. We also evaluate our method against state-of-the-art navigation approaches and professional human drone pilots.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

A General Framework for Uncertainty Estimation in Deep Learning (2019, methodology): “To predict steering angles, we use the DroNet architecture of Loquercio et al. [31], since it was shown to allow closed-loop control on several unmanned aerial platforms [32], [33].”
Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing (2018, methodology): “Recent work has shown that deep networks can provide drones with robust perception capabilities and facilitate safe navigation even in dynamic environments [9], [8].”
Recent Advances in Robot Learning from Demonstration (2020, background): “Further, recent advances in deep learning have fueled the development of end-to-end LfD methods for aerial vehicles (176, 177).”
ANYmal parkour: Learning agile navigation for quadrupedal robots (2023, background): “The locomotion module contains a catalog of locomotion skills that can overcome specific terrains.”
Legged Locomotion in Challenging Terrains using Egocentric Vision (2022, methodology): “This 264 technique has been fruitfully applied to self-driving [80, 81], autonomous drone flight [82, 83], 265 among others.”
Combining Optimal Control and Learning for Visual Navigation in Novel Environments (2019, methodology): “[33, 29] use learning to predict waypoints that are used with model-based control for drone racing.”