ElasticFusion: Real-time dense SLAM and light source estimation

By Thomas Whelan, Renato F. Salas-Moreno, Ben Glocker, Andrew J. Davison, Stefan Leutenegger. 2016

711 citations recorded by the source corpus.

We present a novel approach to real-time dense visual simultaneous localisation and mapping. Our system is capable of capturing comprehensive dense globally consistent surfel-based maps of room scale environments and beyond explored using an RGB-D camera in an incremental online fashion, without pose graph optimization or any post-processing steps. This is accomplished by using dense frame-to-model camera tracking and windowed surfel-based fusion coupled with frequent model refinement through non-rigid surface deformations. Our approach applies local model-to-model surface loop closure optimizations as often as possible to stay close to the mode of the map distribution, while utilizing global loop closure to recover from arbitrary drift and maintain global consistency. In the spirit of improving map quality as well as tracking accuracy and robustness, we furthermore explore a novel approach to real-time discrete light source detection. This technique is capable of detecting numerous light sources in indoor environments in real-time as a user handheld camera explores the scene. Absolutely no prior information about the scene or number of light sources is required. By making a small set of simple assumptions about the appearance properties of the scene our method can incrementally estimate both the quantity and location of multiple light sources in the environment in an online fashion. Our results demonstrate that our technique functions well in many different environments and lighting configurations. We show that this enables (a) more realistic augmented reality rendering; (b) a richer understanding of the scene beyond pure geometry and; (c) more accurate and robust photometric tracking.

Subfields: Aerospace Engineering, Environmental Engineering, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Environmental Engineering

In Geology

Citations in the corpus

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes (2017, background): “There is a large variety of algorithms targeting this scenario [59, 88, 7, 62, 37, 89, 42, 9, 90, 38, 12].”
2D Gaussian Splatting for Geometrically Accurate Radiance Fields (2024, methodology): “They are widely used in SLAM [Whelan et al. 2016] and other robotics tasks [Schöps et al. 2019] as an efficient geometry representation.”
Deferred neural rendering (2019, background): “…with commodity RGB-D sensors, researchers have made significant progress, enabling robust tracking [Choi et al. 2015; Dai et al. 2017; Izadi et al. 2011; Newcombe et al. 2011; Whelan et al. 2016] and large-scale 3D scene capture [Chen et al. 2013; Nießner et al. 2013; Zeng et al. 2013].”
Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM (2019, methodology): “ElasticFusion combines the merits of dense reconstruction and globally consistent mapping by using a deformable model [21].”
MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM (2018, methodology): “In camera tracking, we weighted photometric and geometric terms by their measurement uncertainty, instead of a single weight such as in [17].”
Global Pose Estimation with an Attention-Based Recurrent Network (2018, methodology): “Some methods focus on dense recontruction of the scene, for instance [47] builds dense globally consistent surfel-based maps of room scale environments explored using an RGB-D camera, without pose graph optimisation, while KinectFusion [35] obtains depth measurements directly using active sensors an”
Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations (2018, methodology): “In particular, we consider SemanticFusion [1], where the SLAM reconstruction is carried out by ElasticFusion [53], which relies on RGB-D inputs in order to find dense correspondences between frames.”
Colored Point Cloud Registration Revisited (2017, background): “[41], a state-of-the-art real-time pipeline.”
RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation (2018, result): “(cid:129) ElasticFusion (Whelan, Salas ‐ Moreno, Glocker, Davison, & Leuteneg-ger, 2016), Kintinuous (Whelan et al., 2015), BundleFusion (Dai et al., 2017), and InfiniTAM (Kähler, Prisacariu & Murray, 2016) All these previous visual SLAM approaches assume that the camera is never obstructed or that ”
ConceptFusion: Open-set Multimodal 3D Mapping (2023, methodology): “At the core of nearly all of these methods is the volumetric fusion technique from Curless and Levoy [28], which has been adapted for real-time incremental capture of surface geometry and color [29, 30, 31, 32, 33].”
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras (2016, methodology): “[15] builds a surfel-based map of the environment.”
Neural Autonomous Navigation with Riemannian Motion Policy (2019, background): “Although significant progress has been made [8], [9], [10], [11], these techniques still have trouble with textureless environments or fast-moving cameras.”
Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM (2023, background): “Subsequent works focused on addressing the scalability issue by adopting more efficient data structures, such as surface elements (surfels) [35, 36], VoxelHashing [3, 9, 19] or Octrees [31,41].”
Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization (2022, neutral): “Stückler and Behnke [58] and Whelan et al. [66] optimize a map of surfels , to circumvent the need to correct structured representations ( e.g., meshes or voxels).”