Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization

By Simon Lynen, Torsten Sattler, M. Bosse, Joel A. Hesch, Marc Pollefeys, Roland Yves Siegwart. 2015

262 citations recorded by the source corpus.

Accurately estimating a robot's pose relative to a global scene model and precisely tracking the pose in real-time is a fundamental problem for navigation and obstacle avoidance tasks. Due to the computational complexity of localization against a large map and the memory consumed by the model, state-of-the-art approaches are either limited to small workspaces or rely on a server-side system to query the global model while tracking the pose locally. The latter approaches face the problem of smoothly integrating the server's pose estimates into the trajectory computed locally to avoid temporal discontinuities. In this paper, we demonstrate that large-scale, real-time pose estimation and tracking can be performed on mobile platforms with limited resources without the use of an external server. This is achieved by employing map and descriptor compression schemes as well as efficient search algorithms from computer vision. We derive a formulation for integrating the global pose information into a local state estimator that produces much smoother trajectories than current approaches. Through detailed experiments, we evaluate each of our design choices individually and document its impact on the overall system performance, demonstrating that our approach outperforms state-of-the-art algorithms for localization at scale.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Drone Racing: Learning Agile Flight in Dynamic Environments (2018, background): “Additionally, state-of-the-art state estimation pipelines may require expensive sensors [2], have high computational costs [22], or be subject to drift due to the use of compressed maps [18].”
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016, neutral): “Some initial solutions have been recently proposed for localization against a compressed known map [163], and for memory-efficient dense reconstruction [136].”
Maplab: An Open Framework for Research in Visual-Inertial Mapping and Localization (2017, unknown): “The algorithm is a (partial) implementation of [36].”
Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback (2017, background): “However, all odometry frameworks inherently suffer from drift and, if the primary goal is localization accuracy, a back-end framework doing global mapping, re-localization and loop closure will be indispensable (e.g. Lynen et al. (2015)).”
Visual-Inertial Navigation: A Concise Review (2019, methodology): “[110] developed a largescale map-based VINS that uses a compressed prior map containing feature positions and their uncertainties and employs the matches to features in the prior map to constrain the estimates globally.”
Privacy Preserving Image Queries for Camera Localization (2019, methodology): “Structure-based camera pose estimation methods are now quite mature [28,38,40,54,79] and deployed in many products (e.”
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression (2019, unknown): “Descriptor-based methods handle city-scale scenes [37, 39, 68, 81] and run in real-time on mobile devices [6, 38, 41, 48].”
Image-Based Localization Using LSTMs for Structured Feature Correlation (2016, background): “The two datasets are very different from each other: 7Scenes has a very high number of images in a very small spatial extent, hence, it is more suited for applications such as Augmented Reality, while Cambridge Landmarks has sparser coverage and larger spatial extent, the perfect scenario for image-”
LDSO: Direct Sparse Odometry with Loop Closure (2018, methodology): “[8] proposes to directly include 2D-3D matches from an existing map as Kalman Filter updates in a local MSCKF-style odometry estimator.”
Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions (2018, background): “Work on 3D structure-based visual localization has mainly focused on increasing efficiency [35, 42, 56], im-”
Privacy Preserving Image-Based Localization (2019, methodology): “This approach has been widely used [33,45,46,59, 84] and enables efficient and accurate image-based localization in large scenes.”
Visual-Inertial Monocular SLAM With Map Reuse (2016, background): “The recent system [10] is able to reuse a given map, built offline, and perform visual-inertial tracking.”
From Coarse to Fine: Robust Hierarchical Localization at Large Scale (2018, background): “These improve the runtime on mobile devices but further impair the robustness of the localization, limiting their operations to stable conditions [27].”