Direct visual-inertial odometry with stereo cameras

By Vladyslav C. Usenko, Jakob J. Engel, Jörg Stückler, Daniel Cremers. 2016

228 citations recorded by the source corpus.

We propose a novel direct visual-inertial odometry method for stereo cameras. Camera pose, velocity and IMU biases are simultaneously estimated by minimizing a combined photometric and inertial energy functional. This allows us to exploit the complementary nature of vision and inertial data. At the same time, and in contrast to all existing visual-inertial methods, our approach is fully direct: geometry is estimated in the form of semi-dense depth maps instead of manually designed sparse keypoints. Depth information is obtained both from static stereo - relating the fixed-baseline images of the stereo camera - and temporal stereo - relating images from the same camera, taken at different points in time. We show that our method outperforms not only vision-only or loosely coupled approaches, but also can achieve more accurate results than state-of-the-art keypoint-based methods on different datasets, including rapid motion and significant illumination changes. In addition, our method provides high-fidelity semi-dense, metric reconstructions of the environment, and runs in real-time on a CPU.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing (2018, methodology): “Traditional approaches to autonomous MAV navigation build on visual inertial odometry (VIO) [5], [1], [13], [24] or simultaneous localization and mapping (SLAM) [21], [23], which are used to provide a pose estimate of the drone relative to an internal metric map [14], [4].”
The TUM VI Benchmark for Evaluating Visual-Inertial Odometry (2018, unknown): “Variants include monocular ([1], [2]), stereo ([3], [4]) and visual-inertial ([5], [6], [7]) methods.”
Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback (2017, methodology): “Depending on the experimental setup, state-of-the-art algorithms reduce position errors to 0.1% of the traveled distance (Leutenegger et al. (2015); Forster et al. (2016); Usenko et al. (2016)).”
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator (2017, background): “Direct approaches [2], [3], [22] minimize photometric error, while indirect approaches [12], [15], [17] minimize the geometric displacement.”
Visual-Inertial Navigation: A Concise Review (2019, methodology): “In contrast, direct methods [96, 113, 114, 115] utilize raw pixel intensities in their formulation and allow for inclusion of a larger percentage of the available image information.”
Factor Graphs for Robot Perception (2017, unknown): “[193] combine a semi-direct odometry algorithm with inertial information, proposing a factor graph approach that essentially follows the approach in [103].”
Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight (2017, methodology): “Only a few VIO solutions are designed for stereo or multicamera system [4], [16], [26], [27] compared to the vast amount of work on monocular systems.”
Visual-Inertial Monocular SLAM With Map Reuse (2016, methodology): “the use of the marginalization of past states to maintain a constant computational cost [1], [2], [5]–[8], or the use of full smoothing [3], with an almost constant complexity in exploration but that can be as expensive as a batch method in the presence of loop closures [9].”
Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras (2017, background): “Since then, the semi-dense direct VO/SLAM has been further extended by support for omnidirectional cameras [1] and tightly coupling with IMU [28].”