ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals

By Emanuele Palazzolo, Jens Behley, Philipp Lottes, Philippe Giguère, Cyrill Stachniss. IEEE/RJS International Conference on Intelligent RObots and Systems, 2019

397 citations recorded by the source corpus.

Mapping and localization are essential capabilities of robotic systems. Although the majority of mapping systems focus on static environments, the deployment in real-world situations requires them to handle dynamic objects. In this paper, we propose an approach for an RGB-D sensor that is able to consistently map scenes containing multiple dynamic elements. For localization and mapping, we employ an efficient direct tracking on the truncated signed distance function (TSDF) and leverage color information encoded in the TSDF to estimate the pose of the sensor. The TSDF is efficiently represented using voxel hashing, with most computations parallelized on a GPU. For detecting dynamics, we exploit the residuals obtained after an initial registration, together with the explicit modeling of free space in the model. We evaluate our approach on existing datasets, and provide a new dataset showing highly dynamic scenes. These experiments show that our approach often surpass other state-of-the-art dense SLAM methods. We make available our dataset with the ground truth for both the trajectory of the RGB-D sensor obtained by a motion capture system and the model of the static environment using a high-precision terrestrial laser scanner. Finally, we release our approach as open source code.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Geology

View Paper: https://www.semanticscholar.org/paper/f7f3be44f2b42b083e74658f48d8844ccae5ad43

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

Continuous 3D Perception Model with Persistent State (2025, neutral): “Following MonST3R [98], we evaluate monocular depth estimation on KITTI [26], Sin-tel [10], Bonn [54] and NYU-v2 [52] datasets covering dynamic and static, indoor and outdoor scenes.”
SuMa++: Efficient LiDAR-based Semantic SLAM (2019, background): “There exists also a large body of literature tackling localization and mapping changing environments, for example by filtering moving objects [14], considering residuals in matching [22], or by exploiting sequence information [34].”
RigidFusion: RGB‐D Scene Reconstruction with Rigidly‐moving Objects (2021, methodology): “ReFusion [PBL ∗ 19] is an example of robust background reconstruction methods and reaches a F1 of 0.63 on background reconstruction and a MOTA of 66% on camera tracking.”
MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion (2024, methodology): “Similar to DepthCrafter, we evaluate video depth on KITTI (Geiger et al., 2013), Sintel (Butler et al., 2012), and Bonn (Palazzolo et al., 2019) benchmark datasets, covering dynamic and static, indoor and outdoor, and realistic and synthetic data.”