Vision-Controlled Micro Flying Robots: From System Design to Autonomous Navigation and Mapping in GPS-Denied Environments

By Davide Scaramuzza, Michael Achtelik, Lefteris Doitsidis, Friedrich Fraundorfer, Elias B. Kosmatopoulos, Agostino Martinelli, Markus Achtelik, Margarita Chli, Savvas A Chatzichristofis, Laurent Kneip and 12 more. 2014

317 citations recorded by the source corpus.

Autonomous microhelicopters will soon play a major role in tasks like search and rescue, environment monitoring, security surveillance, and inspection. If they are further realized in small scale, they can also be used in narrow outdoor and indoor environments and represent only a limited risk for people. However, for such operations, navigating based only on global positioning system (GPS) information is not sufficient. Fully autonomous operation in cities or other dense environments requires microhelicopters to fly at low altitudes, where GPS signals are often shadowed, or indoors and to actively explore unknown environments while avoiding collisions and creating maps. This involves a number of challenges on all levels of helicopter design, perception, actuation, control, and navigation, which still have to be solved. The Swarm of Micro Flying Robots (SFLY) project was a European Union-funded project with the goal of creating a swarm of vision-controlled microaerial vehicles (MAVs) capable of autonomous navigation, three-dimensional (3-D) mapping, and optimal surveillance coverage in GPS-denied environments. The SFLY MAVs do not rely on remote control, radio beacons, or motion-capture systems but can fly all by themselves using only a single onboard camera and an inertial measurement unit (IMU). This article describes the technical challenges that have been faced and the results achieved from hardware design and embedded programming to vision-based navigation and mapping, with an overview of how all the modules work and how they have been integrated into the final system. Code, data sets, and videos are publicly available to the robotics community. Experimental results demonstrating three MAVs navigating autonomously in an unknown GPS-denied environment and performing 3-D mapping and optimal surveillance coverage are presented.

Subfields: Aerospace Engineering, Computer Networks and Communications, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

Citations in the corpus

A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones (2018, methodology): “…of a UAV is the so-called localization-mapping-planning cycle, which consists of estimating the robot motion using either offboard (e.g., GPS [15]) or onboard sensors (e.g., visual-inertial sensors [16]), building a local 3D map of the environment, and planning a safe trajectory through it [10].”
Relaxed hover solutions for multicopters: Application to algorithmic redundancy and novel vehicles (2016, background): “…with six or more propellers, instead of a four-propeller quadrocopter, is that the vehicle is able to maintain normal flight if one of the propellers fails (see, for example, Scaramuzza et al. (2014) for a hexacopter design and Marks et al. (2012) for an octocopter rotor failure strategy).”
Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness (2017, background): “[29] also combine IMU and three cameras for obstacle avoidance and map building.”
Autonomous, Vision‐based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle (2016, methodology): “The most similar to our system is the one which resulted from the European project SFLY Weiss et al. (2013); Scaramuzza et al. (2014). However, in SFLY, dense 3D maps were computed offline and were available only after several minutes after landing.”
The current state and future outlook of rescue robotics (2019, background): “Throughout the project, the member labs have made fundamental contributions in perception (Fankhauser et al., 2018; Gawel et al., 2017; Scaramuzza et al., 2014), control (Bellicoso, Jenelten, Gehring, & Hutter, 2018b; Faessler et al.”
Optic-Flow Based Control of a 46g Quadrotor (2013, background): “One of the most successful solution is inertial SLAM (Simultaneous Localization and Mapping) [2], [3], [4], [5],”
Automatic re-initialization and failure recovery for aggressive flight with a monocular vision-based quadrotor (2015, background): “…by the Swiss National Science Foundation through project number 200021-143607 (Swarm of Flying Cameras) and the National Centre of Competence in Research Robotics (NCCR). control) are required by the operators, which renders reinitialization during flight very difficult or even impossible.”
Autonomous aerial navigation using monocular visual‐inertial fusion (2018, unknown): “= 1 kλmin H [ u 1 ] + J (19) = 1 kλmin ⎢⎢⎢⎣ −h1− −h2− −h3− ⎥⎥⎥⎦ [ u 1 ] + ⎢⎢⎢⎣ J1 J2 J3 ⎥⎥⎥⎦ (20)”
SVO: Fast semi-direct monocular visual odometry (2014, background): “This setup allowed fully autonomous way-point following in outdoor areas [1]–[3] and collaboration between MAVs and ground robots [4], [5].”
ORB-SLAM: A Versatile and Accurate Monocular SLAM System (2015, methodology): “While our current implementation makes use of ORB, the techniques proposed are not restricted to these features.”