Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing

By Elia Kaufmann, Mathias Gehrig, Philipp Foehn, René Ranftl, Alexey Dosovitskiy, Vladlen Koltun, Davide Scaramuzza. 2018

136 citations recorded by the source corpus.

Autonomous micro aerial vehicles still struggle with fast and agile maneuvers, dynamic environments, imperfect sensing, and state estimation drift. Autonomous drone racing brings these challenges to the fore. Human pilots can fly a previously unseen track after a handful of practice runs. In contrast, state-of-the-art autonomous navigation algorithms require either a precise metric map of the environment or a large amount of training data collected in the track of interest. To bridge this gap, we propose an approach that can fly a new track in a previously unseen environment without a precise map or expensive data collection. Our approach represents the global track layout with coarse gate locations, which can be easily estimated from a single demonstration flight. At test time, a convolutional network predicts the poses of the closest gates along with their uncertainty. These predictions are incorporated by an extended Kalman filter to maintain optimal maximum-a-posteriori estimates of gate locations. This allows the framework to cope with misleading high-variance estimates that could stem from poor observability or lack of visible gates. Given the estimated gate poses, we use model predictive control to quickly and accurately navigate through the track. We conduct extensive experiments in the physical world, demonstrating agile and robust flight through complex and diverse previously-unseen race tracks. The presented approach was used to win the IROS 2018 Autonomous Drone Race Competition, outracing the second-placing team by a factor of two.

Subfields: Aerospace Engineering, Automotive Engineering, Computer Vision and Pattern Recognition, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

A General Framework for Uncertainty Estimation in Deep Learning (2019, background): “[20] demonstrated the possibility to combine optimal control methods to a network-based perception system by using uncertainty estimation and filtering.”
Combining Optimal Control and Learning for Visual Navigation in Novel Environments (2019, background): “for high-speed navigation [27, 28, 29, 30, 31]. Drews et al. [28, 32] learn a cost function from monocular images for aggressive race-track driving via Model Predictive Control (MPC). Kaufmann et al. [33, 29] use learning to predict waypoints that are used with model-based control for drone racing. ”
The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark Towards Physically Realistic Embodied AI (2021, methodology): “Current state-of-the-art methods on embodied tasks mainly fall into two broad categories: end-to-end training of neural policies using RL [52, 26] or hierarchical RL [24, 2] and map building for path planning [20, 30, 7].”
Learning to Explore using Active Neural SLAM (2020, background): “For example in the context of navigation, Bansal et al. (2019) and Kaufmann et al. (2019) design modular policies for navigation, that interface learned policies with low-level feedback controllers.”
ReLMoGen: Integrating Motion Generation in Reinforcement Learning for Mobile Manipulation (2021, background): “Researchers have proposed learning-based solutions that map partial observations to waypoints [22, 23, 24, 25] or trajectories [26], thus bypassing the trajectory searching problem.”
ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation (2020, background): “Researchers have proposed learning-based solutions that map partial observations to either waypoints [22, 23, 24, 25] or whole trajectories [26], thus bypassing the trajectory searching problem.”