Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds

By Francis Engelmann, Theodora Kontogianni, Alexander Hermans, Bastian Leibe. 2017

281 citations recorded by the source corpus.

Deep learning approaches have made tremendous progress in the field of semantic segmentation over the past few years. However, most current approaches operate in the 2D image space. Direct semantic segmentation of unstructured 3D point clouds is still an open research problem. The recently proposed PointNet architecture presents an interesting step ahead in that it can operate on unstructured point clouds, achieving encouraging segmentation results. However, it subdivides the input points into a grid of blocks and processes each such block individually. In this paper, we investigate the question how such an architecture can be extended to incorporate larger-scale spatial context. We build upon PointNet and propose two extensions that enlarge the receptive field over the 3D scene. We evaluate the proposed strategies on challenging indoor and outdoor datasets and show improved results in both scenarios.

Subfields: Computational Mechanics, Environmental Engineering, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Environmental Engineering

In Geology

Citations in the corpus

Dynamic Graph CNN for Learning on Point Clouds (2018, methodology): “seline, where additional point features (local point density, local curvature and normal) are used to construct handcrafted features and then fed to an MLP classiﬁer. We further compare our work with [12], who present network architectures to enlarge the receptive ﬁeld over the 3D scene. Two differe”
DeepGCNs: Can GCNs Go As Deep As CNNs? (2019, background): “More recent work focuses on directly processing unordered point cloud representations [28, 31, 9, 15, 50].”
Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions (2020, background): “An active research area within 3D scene understanding focuses on 3D semantic segmentation [34, 4, 36, 7, 38, 12] and object detection and classification [41, 34, 35, 52].”
Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs (2017, methodology): “[10] improves on this by increasing the context scope with multi-scale windows or by considering directly neighboring window positions on a voxel grid.”
On Universal Equivariant Set Networks (2019, unknown): “(2018) and Engelmann et al. (2017). PointNsetSeg uses an invariant version of PointNet (i.”
OpenMask3D: Open-Vocabulary 3D Instance Segmentation (2023, unknown): “Given a 3D scene as input, 3D semantic segmentation task aims to assign a semantic category to each point in the scene [2–4, 9, 14, 15, 22, 28, 29, 31, 38, 42, 44, 46, 47, 53, 54, 63, 64, 66, 68, 70, 73].”
Associatively Segmenting Instances and Semantics in Point Clouds (2019, methodology): “PointNet [26] and following works [7, 38] use multilayer perceptron (MLP) to produce fine-grained point-level”
Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds (2019, background): “It is desirable to directly train and test on large-scale input point clouds instead of the divided small blocks, by drawing on the recent work [10][22].”