Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping

By Antoni Rosinol, Marcus Abate, Yun Chang, Luca Carlone. IEEE International Conference on Robotics and Automation, 2019

637 citations recorded by the source corpus.

We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM). The library goes beyond existing visual and visual-inertial SLAM libraries (e.g., ORB-SLAM, VINS-Mono, OKVIS, ROVIO) by enabling mesh reconstruction and semantic labeling in 3D. Kimera is designed with modularity in mind and has four key components: a visual-inertial odometry (VIO) module for fast and accurate state estimation, a robust pose graph optimizer for global trajectory estimation, a lightweight 3D mesher module for fast mesh reconstruction, and a dense 3D metric-semantic reconstruction module. The modules can be run in isolation or in combination, hence Kimera can easily fall back to a state-of-the-art VIO or a full SLAM system. Kimera runs in real-time on a CPU and produces a 3D metric-semantic mesh from semantically labeled images, which can be obtained by modern deep learning methods. We hope that the flexibility, computational efficiency, robustness, and accuracy afforded by Kimera will build a solid basis for future metric-semantic SLAM and perception research, and will allow researchers across multiple areas (e.g., VIO, SLAM, 3D reconstruction, segmentation) to benchmark and prototype their own efforts without having to start from scratch.

View Paper: https://www.semanticscholar.org/paper/86db02456021ecbd3d813ecd38b52c01a364a16c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM (2020, methodology): “Many recent visual SLAM and VO systems [2], [7], [8] solve place recognition using the DBoW2 bag of words library [9].”
Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems (2021, methodology): “KimeraSemantics [16] processes depth images (possibly obtained from RGB-D cameras or by stereo matching) and 2D semantic segmentations [76] and produces a dense metric-semantic 3D mesh using the VIO pose estimates.”
Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning (2024, methodology): “We also observe the performance of our method exceeds the VIO system Kimera [26] and iS LAM [12], both with stereo-inertial settings.”
Scan Context++: Structural Place Recognition Robust to Rotation and Lateral Variations in Urban Environments (2021, background): “CC outperformed the others in challenging regions with few false positives (red), which can potentially be treated using existing robust back-ends [45, 48, 49].”
Continual Learning of Semantic Segmentation using Complementary 2D-3D Data Representations (2021, methodology): “Prior work focuses on accumulation of 2D semantic network predictions in a global 3D representation to enhance the semantic predictions [18, 12, 25, 3].”
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation (2022, methodology): “Kimera [48] takes a stereo sequence and does online reconstruction, meshing, and semantic labeling of the mesh using ground-truth labels, as a proxy for any 2D segmentation method.”
Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization (2022, unknown): “For timing of the low-level processes in Hydra, we refer the reader to the analysis in [50], as we also rely on Kimera-VIO.”
DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras (2021, methodology): “Indirect approaches [31, 32, 5, 37] first process the image into an intermediate representation by detecting points of interest and attaching feature descriptors.”
KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D (2021, background): “semantic reconstructions of SLAM algorithms at roomlevel [87], [102], existing works on outdoor semantic SLAM typically evaluate only pose estimation while ignoring the quality of the semantic reconstruction [12], [23].”
Atlas: End-to-End 3D Scene Reconstruction from Posed Images (2020, background): “A notable exception is Kimera [41], which uses multiview stereo [23] to predict depth, however, they only show results on synthetic data and ground truth 2D segmentations.”