Learning high-speed flight in the wild

By Antonio Loquercio, Elia Kaufmann, René Ranftl, Matthias Müller, Vladlen Koltun, Davide Scaramuzza. Science Robotics, 2021

449 citations recorded by the source corpus.

Description Deep Learning enables agile flight in challenging environments with onboard sensing and computation. Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. Although this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. The subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline. Here, we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and human-made environments at high speeds with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains, and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Ocean Engineering

View Paper: https://www.semanticscholar.org/paper/948216c6a567e6cf01882409e75e9957a55ddb1f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Ocean Engineering

Citations in the corpus

Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “High-speed quadrotor control in outdoor scenarios with complex obstacles further requires the policy to ensure the long-horizon feasibility of the closed-loop trajectories (72).”
Vision-Only Robot Navigation in a Neural Radiance World (2021, background): “While some existing vision-only navigation systems [2] have seen recent success with end-to-end approaches, others such as [3] have advocated for a modularization of learned perception and control and achieved impressive results.”
Neural-Fly enables rapid learning for agile flight in strong winds (2022, methodology): “Note that our method supports any trajectory planners such as [1] or learning-based planners [47, 48, 49].”