Real-time large-scale dense RGB-D SLAM with volumetric fusion

By Thomas Whelan, Michael Kaess, Hordur Johannsson, Maurice F. Fallon, John J. Leonard, John McDonald Jr.. Int. J. Robotics Res., 2014

497 citations recorded by the source corpus.

We present a new simultaneous localization and mapping (SLAM) system capable of producing high-quality globally consistent surface reconstructions over hundreds of meters in real time with only a low-cost commodity RGB-D sensor. By using a fused volumetric surface reconstruction we achieve a much higher quality map over what would be achieved using raw RGB-D point clouds. In this paper we highlight three key techniques associated with applying a volumetric fusion-based mapping system to the SLAM problem in real time. First, the use of a GPU-based 3D cyclical buffer trick to efficiently extend dense every-frame volumetric fusion of depth maps to function over an unbounded spatial region. Second, overcoming camera pose estimation limitations in a wide variety of environments by combining both dense geometric and photometric camera pose constraints. Third, efficiently updating the dense map according to place recognition and subsequent loop closure constraints by the use of an ‘as-rigid-as-possible’ space deformation. We present results on a wide variety of aspects of the system and show through evaluation on de facto standard RGB-D benchmarks that our system performs strongly in terms of trajectory estimation, map quality and computational performance in comparison to other state-of-the-art systems.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Geology

View Paper: https://www.semanticscholar.org/paper/daf19f6e6564f7de43dca62ca67a8f9b9f69d067

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation (2018, background): “• ElasticFusion (Whelan, Salas‐Moreno, Glocker, Davison, & Leutenegger, 2016), Kintinuous (Whelan et al., 2015), BundleFusion (Dai et al.”
Gaussian Splatting SLAM (2023, unknown): “Classical map-centric approaches often use voxel grids [2, 23, 26, 40] or points [9, 29, 41] as the underlying 3D representation.”
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (2023, methodology): “Traditional approaches to dense SLAM have explored a variety of explicit representations, including 2.5D images [15, 34], (truncated) signed-distance functions [6, 25, 26, 42], gaussian mixture models [9, 10] and circular sur-fels [13, 33, 43].”
Visual Place Recognition: A Survey (2016, background): “While many systems use data from additional sensors such as lasers [98] or RGB-D cameras [126]–[128], geometric data can also be extracted from conventional cameras to allow metric calculation of the robot pose.”
Efficient Surfel-Based SLAM using 3D Laser Range Data in Urban Environments (2018, background): “The surfel-based representation allows to model comparably large environments without a compromise in terms of reduced reconstruction fidelity due to reduced grid resolution or a more complex implementation [32, 19].”
Very High Frame Rate Volumetric Integration of Depth Images on Mobile Devices (2015, methodology): “kept in the active memory and other parts get swapped out and converted to a more memory efficient triangle mesh [22, 4, 27].”
Fusion++: Volumetric Object-Level SLAM (2018, background): “56m along the z-axis in the camera frame F − →C to prevent wasted volume as in [37].”
Factor Graphs for Robot Perception (2017, methodology): “[197, 196] present a dense SLAM solution, Kintinuous, that extends the local dense method KinectFusion to large environments, see Figure 7.”
A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation (2016, background): “H IGH-quality 3D scene data has become increasingly available thanks to the growing popularity of consumergrade depth sensors and tremendous progress in 3D scene reconstruction research [1], [2], [3], [4], [5], [6].”
Colored Point Cloud Registration Revisited (2017, methodology): “We build on dense and direct formulations for RGB-D image alignment, which use the two-dimensional image plane as the parameterization domain [35, 25, 44, 40].”
School of Electrical and Computer Engineering Oklahoma State University , Stillwater , OK 74078 , USA (methodology): “In general, researchers in the robotics community deploy Simultaneous Localization and Mapping (SLAM) framework for sparse 3D mapping using visual and depth information, and their ambition is ultimately to minimize the trajectory error [14, 23, 24, 17, 26] with less concern about the pertinent detai”
SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems (2017, background): “When a depth ﬁlter’s uncertainty becomes small enough, a new 3D point is inserted in the map and is immediately used for motion estimation.”
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras (2016, methodology): “Kintinuous by Whelan et al. [12] was able to operate in large environments by using a rolling cyclical buffer and included loop closing using place recognition and pose graph optimization.”
CNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction (2017, methodology): “From the point of view of the type of input data being processed, approaches can be classified into either depth camera-based [21, 30, 11] and monocular camera-based [22, 4, 20].”
Real-Time Large-Scale Dense 3D Reconstruction with Loop Closure (2016, methodology): “Quite the opposite is done in the map-centric representation used in Kintinuous 2.0 [31], ElasticFusion [18] and an earlier work along the same lines [23].”
BAD SLAM: Bundle Adjusted Direct RGB-D SLAM (2019, background): “For example, Kintinuous [68] deforms the reconstructed mesh for loop closing.”
ElasticFusion: Dense SLAM Without A Pose Graph (2015, methodology): “System kt0 kt1 kt2 kt3 DVO SLAM 0.104m 0.029m 0.191m 0.152m RGB-D SLAM 0.026m 0.008m 0.018m 0.433m MRSMap 0.204m 0.228m 0.189m 1.090m Kintinuous 0.072m 0.005m 0.010m 0.355m Frame-to-model 0.497m 0.009m 0.020m 0.”