iMAP: Implicit Mapping and Positioning in Real-Time

By Edgar Sucar, Shikun Liu, Joseph Ortiz, Andrew J. Davison. IEEE International Conference on Computer Vision, 2021

888 citations recorded by the source corpus.

We show for the first time that a multilayer perceptron (MLP) can serve as the only scene representation in a real-time SLAM system for a handheld RGB-D camera. Our network is trained in live operation without prior data, building a dense, scene-specific implicit 3D model of occupancy and colour which is also immediately used for tracking.Achieving real-time SLAM via continual training of a neural network against a live image stream requires significant innovation. Our iMAP algorithm uses a keyframe structure and multi-processing computation flow, with dynamic information-guided pixel sampling for speed, with tracking at 10 Hz and global map updating at 2 Hz. The advantages of an implicit MLP over standard dense SLAM techniques include efficient geometry representation with automatic detail control and smooth, plausible filling-in of unobserved regions such as the back surfaces of objects.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3629e0ba2ffb8406d8aba880685f06aa35e3cd7f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Gaussian Splatting SLAM (2023, positive): “In the RGB-D case, we compare against neural-implicit SLAM meth-ods [7, 8, 27, 33, 39, 43, 46] which are also map-centric, rendering-based and do not perform loop closure.”
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM (2021, methodology): “The modified loss down-weights less certain regions in the reconstructed geometry [46, 62], e.”
Advances in Neural Rendering (2021, methodology): “Beyond the immediate use case of AR/VR, there is little research on using neural scene representations in other contexts like robotics, with the notable exception of the real-time SLAM system iMAP [SLOD21] supporting single-room environments (see Figure 14).”
Urban Radiance Fields (2021, methodology): “In work more similar to ours, IMAP [60] performs real-time SLAM from RGB-D images captured with a hand-held camera moving through indoor scenes from the Replica Dataset [59].”
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (2023, methodology): “For the past three decades, SLAM research has extensively centered around the question of map representation – resulting in a variety of sparse [2, 3, 7, 23], dense [4, 6, 8, 13, 15, 25, 26, 34, 42, 43], and neural scene representations [21, 29, 30, 37, 46, 55].”
A Survey on 3D Gaussian Splatting (2024, neutral): “While recent diffusion-based 4D generation frameworks can synthesize visually compelling dynamics, they often optimize for appearance metrics and may violate basic conservation principles or material constraints.”
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering (2021, methodology): “This has enabled a number of applications, such as novel view synthesis [7, 3, 6], 3D reconstruction [5, 3] semantic segmentation [8, 9], and SLAM [10].”
DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras (2021, methodology): “Previous work has investigated replacing hand-crafted with learned features[13, 7, 29, 26, 35], using neural 3D representations[46, 1, 9, 45, 44, 25, 22], and combining learned energy terms with classical optimization backends[58, 57].”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, methodology): “iMAP [71] and Nice-SLAM [72] offer excellent NeRF based SLAM frameworks which could integrate faster and better quality NeRF models.”
Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation (2022, methodology): “Inspired by traditional volumetric SLAM systems [34] and the successful application of neural implicit representation in parallel tracking and mapping [31], we propose a more efficient hybrid data structure that combines a sparse voxel representation with neural implicit embeddings.”
Vision-Only Robot Navigation in a Neural Radiance World (2021, methodology): “Unlike voxel models, point clouds, or mesh models, they are trained directly on dense photographic images without the traditional feature extraction, matching, and alignment pipeline [4], [5].”
NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields (2022, methodology): “Unfortunately, these approaches are too slow for online inference due to their choice of a large MLP as map representation. iMap[27] and Nice-SLAM[28] subsequently showed how to build accurate 3D reconstructions, without the need for poses, by partially de-coupling pose and depth estimation,…”
MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction (2022, methodology): “For Replica [67] and ScanNet [13], we report Accuracy , Completeness , Chamfer Distance , Precision , Recall , and F-score with a threshold of 5cm following [21, 68, 92].”