NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields

By Antoni Rosinol, John J. Leonard, Luca Carlone. IEEE/RJS International Conference on Intelligent RObots and Systems, 2022

386 citations recorded by the source corpus.

We propose a novel geometric and photometric 3D mapping pipeline for accurate and real-time scene reconstruction from casually taken monocular images. To achieve this, we leverage recent advances in dense monocular SLAM and real-time hierarchical volumetric neural radiance fields. Our insight is that dense monocular SLAM provides the right information to fit a neural radiance field of the scene in real-time, by providing accurate pose estimates and depth-maps with associated uncertainty. Our proposed pipeline achieves better geometric and photometric accuracy than competing approaches (up to 178% better PSNR and 75% better L1 depth), while working in real-time and using only monocular images.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Geology

View Paper: https://www.semanticscholar.org/paper/1fbe23a0d6e45b0bd7861ea33b7e51c27b1a8ef4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Nikhil Varma Keetha
2. Deva Ramanan
3. Jay Karhade
4. Sebastian A. Scherer
5. Jonathon Luiten
6. Gengshan Yang
7. Krishna Murthy Jatavallabhula

In Computer Vision and Pattern Recognition

1. Jonathon Luiten
2. Gengshan Yang
3. Dening Lu
4. Yin Gao
5. Jonathan Li
6. Kyle Gao
7. Linlin Xu
8. Hongjie He

In Geology

1. Deva Ramanan

Citations in the corpus

• SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (2023, background): “For the past three decades, SLAM research has extensively centered around the question of map representation – resulting in a variety of sparse [2, 3, 7, 23], dense [4, 6, 8, 13, 15, 25, 26, 34, 42, 43], and neural scene representations [21, 29, 30, 37, 46, 55].”
• NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, positive): “DS-NeRF was extensively tested on the DTU dataset [26], NeRF dataset [1], and the RedWood-3dscan dataset [85], outperforming benchmark methods such as baseline NeRF [1], pixelNerF [58] and MVSNeRF [57].”