Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras

By Rui Wang, Martin Schwörer, Daniel Cremers. IEEE International Conference on Computer Vision, 2017

374 citations recorded by the source corpus.

We propose Stereo Direct Sparse Odometry (Stereo DSO) as a novel method for highly accurate real-time visual odometry estimation of large-scale environments from stereo cameras. It jointly optimizes for all the model parameters within the active window, including the intrinsic/extrinsic camera parameters of all keyframes and the depth values of all selected pixels. In particular, we propose a novel approach to integrate constraints from static stereo into the bundle adjustment pipeline of temporal multi-view stereo. Real-time optimization is realized by sampling pixels uniformly from image regions with sufficient intensity gradient. Fixed-baseline stereo resolves scale drift. It also reduces the sensitivities to large optical flow and to rolling shutter effect which are known shortcomings of direct image alignment methods. Quantitative evaluation demonstrates that the proposed Stereo DSO outperforms existing state-of-the-art visual odometry methods both in terms of tracking accuracy and robustness. Moreover, our method delivers a more precise metric 3D reconstruction than previous dense/semi-dense direct approaches while providing a higher reconstruction density than feature-based methods.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/63a5577877ac595d2ca062f260384d71c61e7cb1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Self-Supervised Monocular Depth Hints (2019, methodology): “[35] train a monocular depth estimation network with both self-supervision from stereo pairs, and supervision from sparse depths estimated in batch by the Stereo DSO [33] algorithm.”
BARF: Bundle-Adjusting Neural Radiance Fields (2021, background): “Direct methods, on the other hand, do not rely on such distinctive keypoints — every pixel can contribute to maximizing photometric consistency, leading to improved robustness in sparsely textured environments [59].”
Extending Monocular Visual Odometry to Stereo Camera Systems by Scale optimization (2019, methodology): “Stereo DSO [15] solved the scale problem by using stereo”
Neural Autonomous Navigation with Riemannian Motion Policy (2019, background): “Although significant progress has been made [8], [9], [10], [11], these techniques still have trouble with textureless environments or fast-moving cameras.”
Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation (2019, methodology): “Opposite to our approach, the deep model in [37] is trained using supervision derived by Stereo Direct Sparse Odometry [34].”
Deep Visual Teach and Repeat on Path Networks (2018, background): “Numerous approaches have been suggested that incorporate depth-sensitive sensors (stereo, RGBD, Lidar) [21, 36].”
A Review of Visual-LiDAR Fusion based Simultaneous Localization and Mapping (2020, methodology): “In this category, the most known methods are DTAM [25], LSD-SLAM [26], SVO [27], or DSO [28,29].”
Visual-Inertial Navigation: A Concise Review (2019, background): “Other popular direct methods include [118] and [119] which estimate keyframe depths along with the camera poses in a tightly-coupled manner, offering low-drift results.”
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry (2020, methodology): “…three estimations tightly into both the front-end tracking and the back-end non-linear optimization of a sparse direct odometry framework [16]. competitive performance compared to stereo [53, 74] or visual-inertial odometry (VIO) [44, 54, 56, 72], due to the scale drift [62,77] and low robustness.”
The TUM VI Benchmark for Evaluating Visual-Inertial Odometry (2018, neutral): “Variants include monocular ([1], [2]), stereo ([3], [4]) and visual-inertial ([5], [6], [7]) methods.”
Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry (2018, methodology): “To further improve the accuracy of DVSO, inspired by Stereo DSO [40] which couples the static stereo term with the temporal multi-view energy function, we introduce a novel virtual stereo term E † p for each point p where is the virtual projected coordinate of p us-ing the vector t b denoting the…”
ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM (2020, methodology): “Extensions of this work include stereo [29], loop-closing using features and DBoW2 [59] [60], and visual-inertial odometry [46].”
LDSO: Direct Sparse Odometry with Loop Closure (2018, background): “On the KITTI Odometry Dataset, as shown previously in the thorough evaluation in [19], monocular VO systems like DSO and ORB-SLAM (the VO component only) suffer severe accumulated drift which makes them not usable for such large-scale scenarios.”