Vision-based autonomous mapping and exploration using a quadrotor MAV

By Friedrich Fraundorfer, Lionel Heng, Dominik Honegger, Gim Hee Lee, Lorenz Meier, Petri Tanskanen, Marc Pollefeys. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012

352 citations recorded by the source corpus.

In this paper, we describe our autonomous vision-based quadrotor MAV system which maps and explores unknown environments. All algorithms necessary for autonomous mapping and exploration run on-board the MAV. Using a front-looking stereo camera as the main exteroceptive sensor, our quadrotor achieves these capabilities with both the Vector Field Histogram+ (VFH+) algorithm for local navigation, and the frontier-based exploration algorithm. In addition, we implement the Bug algorithm for autonomous wall-following which could optionally be selected as the substitute exploration algorithm in sparse environments where the frontier-based exploration under-performs. We incrementally build a 3D global occupancy map on-board the MAV. The map is used by the VFH+ and frontier-based exploration in dense environments, and the Bug algorithm for wall-following in sparse environments. During the exploration phase, images from the front-looking camera are transmitted over Wi-Fi to the ground station. These images are input to a large-scale visual SLAM process running off-board on the ground station. SLAM is carried out with pose-graph optimization and loop closure detection using a vocabulary tree. We improve the robustness of the pose estimation by fusing optical flow and visual odometry. Optical flow data is provided by a customized downward-looking camera integrated with a microcontroller while visual odometry measurements are derived from the front-looking stereo camera. We verify our approaches with experimental results.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ca1409706707ebf432234f896f0a45ecb70850a9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications (2013, methodology): “A mouse sensor based optical flow module for quadrotor control is shown in [3].”
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks (2020, background): “Planning algorithms on existing maps [14,46,52] and combined mapping & planning [26,50,49,30,6] are other related research directions.”
Survey on Computer Vision for UAVs: Current Developments and Trends (2017, methodology): “In [50] a vision based UAV system for unknown environment mapping and exploration using a frontlooking stereo camera and a down-looking optic flow camera was presented.”
Deep Neural Network for Real-Time Autonomous Indoor Navigation (2015, background): “Stereo Vision : Accurate depth estimation and relative position estimation are possible using stereo cameras ([5],[6]).”
Learning to fly by crashing (2017, background): “One can also use stereo vision based estimation [12], [13] with light and cheap cameras.”
Learning monocular reactive UAV control in cluttered natural environments (2012, background): “Accurate depth estimation and localization is also possible with visual sensors using stereo cameras [16] or in a moving monocular setup [17].”
Advanced closed loop visual navigation for micro aerial vehicles (2014, methodology): “Fraundorfer et al. (2012) perform dense 3D mapping and obstacle avoidance with an onboard stereo camera and full onboard computation, while the optical flow sensor presented by Honegger et al. (2012) was used for navigation and control.”
Stereo vision-based obstacle avoidance for micro air vehicles using disparity space (2014, methodology): “Stereo vision can provide 3D perception around the focus of expansion whether the aircraft is moving or not, but so far with relatively short lookahead distance [7].”
Stereo vision based indoor/outdoor navigation for flying robots (2013, result): “Our system is most closely related to the work of Fraun-dorfer et al. [10].”
PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition (2018, background): “Localization addresses the question of “where am I in a given reference map”, and it is of paramount importance for robots such as self-driving cars [13] and drones [11] to achieve full autonomy.”
Vision-based state estimation for autonomous rotorcraft MAVs in complex environments (2013, background): “Stereo vision-based state estimation approaches for autonomous MAVs such as those proposed in [2, 9, 10] do not suffer from the problem scale drift as seen in monocular systems or limit the observable camera motion.”
Two Body Problem: Collaborative Visual Task Completion (2019, background): “Combinations of joint mapping and planning have also been discussed [27, 50, 49, 31, 3].”
Autonomous aerial navigation using monocular visual‐inertial fusion (2018, methodology): “xk = [ pwbk , v w bk , qwbk , b b a, b b g ] , k ∈ [0, n] (3)”
Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness (2017, background): “Fraundorfer et al. [11] fuse a forward-facing stereo pair of cameras with a downward-facing camera for map building and state estimation.”
Relaxed hover solutions for multicopters: Application to algorithmic redundancy and novel vehicles (2016, background): “Multicopters have found broad use as research platforms, for example, for vision-based pose estimation with quadrocopters (Fraundorfer et al., 2012) and hexacopters (Weiss et al., 2013), and also as platforms allowing for new capabilities.”
Cognitive Mapping and Planning for Visual Navigation (2017, background): “Works such as [18, 19] have studied the joint problem of mapping and planning.”
Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search (2015, neutral): “This challenge is conventionally addressed by using localization and mapping algorithms to map out the environment [9] and then optimizing trajectories under the resulting map [11].”
Stability and control of a quadrocopter despite the complete loss of one, two, or three propellers (2014, background): “Multicopters have found broad use as research platforms, used e.g. for vision based pose estimation with quadrocopters [1] and hexacopters [2], and also as platforms allowing for new capabilities.”