Large-scale direct SLAM with stereo cameras

By Jakob J. Engel, Jörg Stückler, Daniel Cremers. IEEE/RJS International Conference on Intelligent RObots and Systems, 2015

603 citations recorded by the source corpus.

We propose a novel Large-Scale Direct SLAM algorithm for stereo cameras (Stereo LSD-SLAM) that runs in real-time at high frame rate on standard CPUs. In contrast to sparse interest-point based methods, our approach aligns images directly based on the photoconsistency of all high-contrast pixels, including corners, edges and high texture areas. It concurrently estimates the depth at these pixels from two types of stereo cues: Static stereo through the fixed-baseline stereo camera setup as well as temporal multi-view stereo exploiting the camera motion. By incorporating both disparity sources, our algorithm can even estimate depth of pixels that are under-constrained when only using fixed-baseline stereo. Using a fixed baseline, on the other hand, avoids scale-drift that typically occurs in pure monocular SLAM.We furthermore propose a robust approach to enforce illumination invariance, capable of handling aggressive brightness changes between frames - greatly improving the performance in realistic settings. In experiments, we demonstrate state-of-the-art results on stereo SLAM benchmarks such as Kitti or challenging datasets from the EuRoC Challenge 3 for micro aerial vehicles.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d952091f5d0acfae80bfebb509fc2d3504a519c7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras (2016, background): “The recent Stereo LSD-SLAM of Engel et al. [11] is a semi-dense direct approach that minimizes photometric error in image regions with high gradient.”
Planning and Decision-Making for Autonomous Vehicles (2018, background): “0 (52), and LSD-SLAM (Large-Scale Direct Monocular SLAM) (53), have matured to be able to compete.”
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017, methodology): “Another method to reduce drift is simultaneous localization and mapping (SLAM) (Lee et al. (2013a); Engel et al. (2015); Pire et al. (2015); MurArtal et al. (2015)) which jointly estimates the location and a map of the environment to recognize places that have been visited before.”
Direct Sparse Odometry (2016, result): “Note that in contrast to most previous formulations [6], [13], the scalar factor e ai is parametrized logarithmically.”
Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry (2018, methodology): “St. LSD-VO [10] ORB-SLAM2 [31] St. DSO [40] in,vs,lr DVSO Seq.”
Extending Monocular Visual Odometry to Stereo Camera Systems by Scale optimization (2019, background): “Readers are referred to [8] for notations and detailed treatments.”
Visual SLAM algorithms: a survey from 2010 to 2016 (2017, methodology): “LSD-SLAM is extended to stereo cameras and omni-directional cameras [49, 50].”
Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras (2017, background): “The extension of direct monocular SLAM algorithms to large-scale environments was proposed in LSD-SLAM [8, 9].”
Direct visual-inertial odometry with stereo cameras (2016, methodology): “The parameters a and b correct for affine lighting changes between the images and are optimized alongside the pose ξ in an alternating fashion, as described in [5].”
Visual-Inertial Navigation: A Concise Review (2019, background): “This work was later extended from a monocular sensor to stereo and omnidirectional cameras for improved accuracy [116, 117].”
ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM (2020, background): “PTAM [16]–[18] SLAM FAST Pyramid SSD BA Thumbnail - X - - - - Very Good Fair [19] LSD-SLAM [20], [21] SLAM Edgelets Direct PG FABMAP PG - X X - - - Good Good [22] SVO [23], [24] VO FAST+ Hi.”
Large-scale direct SLAM for omnidirectional cameras (2015, background): “A more thorough evaluation of the effect of image resolution, steering the trade-off between accuracy and computational cost for direct SLAM can be found in [25].”
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching (2021, background): “estimating a disparity/depth map from a pair of stereo images, is fundamental to various applications such as autonomous driving [3], robot navigation [1], SLAM [8, 10], etc.”
RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation (2018, neutral): “But still, visual SLAM approaches integrated in ROS are not often tested on autonomous robots: Only SLAM by teleoperation or by a human moving the sensor (Dai, Nießner, Zollöfer, Izadi, & Theobalt, 2017; Engel, Stückler, & Cremers, 2015; Mur ‐ Artal & Tardós, 2017).”
Efficient Surfel-Based SLAM using 3D Laser Range Data in Urban Environments (2018, background): “There has been significant advances in visionbased [6, 7] and RGB-D-based [18, 33, 3] SLAM systems over the past few years.”
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry (2020, methodology): “Engel et al. also ﬁnd similar issues in [18].”