Meaningful maps with object-oriented semantic mapping

By Niko Sünderhauf, Trung T. Pham, Yasir Latif, Michael Milford, Ian David Reid. 2016

228 citations recorded by the source corpus.

For intelligent robots to interact in meaningful ways with their environment, they must understand both the geometric and semantic properties of the scene surrounding them. The majority of research to date has addressed these mapping challenges separately, focusing on either geometric or semantic mapping. In this paper we address the problem of building environmental maps that include both semantically meaningful, object-level entities and point- or mesh-based geometrical representations. We simultaneously build geometric point cloud models of previously unseen instances of known object classes and create a map that contains these object models as central entities. Our system leverages sparse, feature-based RGB-D SLAM, image-based deep-learning object detection and 3D unsupervised segmentation.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

The limits and potentials of deep learning for robotics (2018, neutral): “As an extension of reasoning over individual objects, inference over the geometry of the whole scene is important for robotic vision, and closely related to the problems of object-based mapping or object-based SLAM [19], [86], [98], [114].”
Fusion++: Volumetric Object-Level SLAM (2018, methodology): “Very closely related to ours is work by S¨underhauf et al. [34], who proposed an object-oriented mapping system composed of instances using bounding box detections from a CNN and an unsupervised geometric segmentation algo-rithm using RGB-D data.”
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras (2016, methodology): “We are aware that it has been already used outof-the-box in [23].”
In-Place Scene Labelling and Understanding with Implicit Scene Representation (2021, background): “In semantic mapping systems (e.g. [29, 13, 18]), multiple 2D semantic observations are integrated into a 3D map or target frames to produce a more consistent and accurate semantic segmentation.”
Bayesian Semantic Instance Segmentation in Open Set World (2018, background): “Semantic instance segmentation based scene understanding has recently at-tracted the interest of the ﬁeld [3, 25].”
SuMa++: Efficient LiDAR-based Semantic SLAM (2019, methodology): “Motivated by the advances of deep learning and Convolutional Neural Networks (CNNs) for scene understanding, there have been many semantic SLAM techniques exploiting this information using cameras [5], [31], cameras + IMU data [4], stereo cameras [10], [15], [18], [33], [38], or RGB-D sensors [3], [”