CubeSLAM: Monocular 3-D Object SLAM

By Shichao Yang, Sebastian A. Scherer. IEEE Transactions on robotics, 2018

462 citations recorded by the source corpus.

In this paper, we present a method for single image three-dimensional (3-D) cuboid object detection and multiview object simultaneous localization and mapping in both static and dynamic environments, and demonstrate that the two parts can improve each other. First, for single image object detection, we generate high-quality cuboid proposals from two-dimensional (2-D) bounding boxes and vanishing points sampling. The proposals are further scored and selected based on the alignment with image edges. Second, multiview bundle adjustment with new object measurements is proposed to jointly optimize poses of cameras, objects, and points. Objects can provide long-range geometric and scale constraints to improve camera pose estimation and reduce monocular drift. Instead of treating dynamic regions as outliers, we utilize object representation and motion model constraints to improve the camera pose estimation. The 3-D detection experiments on SUN RGBD and KITTI show better accuracy and robustness over existing approaches. On the public TUM, KITTI odometry and our own collected datasets, our SLAM method achieves the state-of-the-art monocular camera pose estimation and at the same time, improves the 3-D object detection accuracy.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a5828b9883706fdcc6d961da93386dc510086ea9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, background): “Other works have considered monocular RGB only object-level SLAM [30– 32].”
DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM (2020, background): “The authors of CubeSLAM [11] showed impressive results with only a monocular camera by making use of a 3D bounding box proposal generation based on 2D bounding boxes and vanishing points.”
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, background): “and-alone networks which will consume plenty of computing resources in the training and inferring stages. Another one only employs 2D bounding box and properties of a 3D object as the supervised data [35, 4, 22, 44]. In this case, an intuitional idea is to building a deep regression network to predi”
KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D (2021, background): “boxes and instance labels available in our dataset [71], [109].”
Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, unknown): “An agent with autonomous exploration and decision-making capabilities must not only consider which actions to take to explore the environment, but also determine when to stop exploring to answer questions.”