Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark

By Timo Hackel, Nikolay Savinov, Lubor Ladicky, Jan Dirk Wegner, Konrad Schindler, Marc Pollefeys. arXiv.org, 2017

825 citations recorded by the source corpus.

This paper presents a new 3D point cloud classification benchmark data set with over four billion manually labelled points, meant as input for data-hungry (deep) learning methods. We also discuss first submissions to the benchmark that use deep convolutional neural networks (CNNs) as a work horse, which already show remarkable performance improvements over state-of-the-art. CNNs have become the de-facto standard for many tasks in computer vision and machine learning like semantic segmentation or object detection in images, but have no yet led to a true breakthrough for 3D point cloud labelling tasks due to lack of training data. With the massive data set presented in this paper, we aim at closing this data gap to help unleash the full potential of deep learning methods for 3D labelling tasks. Our this http URL data set consists of dense point clouds acquired with static terrestrial laser scanners. It contains 8 semantic classes and covers a wide range of urban outdoor scenes: churches, streets, railroad tracks, squares, villages, soccer fields and castles. We describe our labelling interface and show that our data set provides more dense and complete point clouds with much higher overall number of labelled points compared to those already available to the research community. We further provide baseline method descriptions and comparison between methods submitted to our online system. We hope this http URL will pave the way for deep learning methods in 3D point cloud labelling to learn richer, more general 3D representations, and first submissions after only a few months indicate that this might indeed be the case.

Subfields: Computational Mechanics, Environmental Engineering, Geology

View Paper: https://www.semanticscholar.org/paper/5d9b36e296e6f61177c2f1739a6ca8c553303c09

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Environmental Engineering

In Geology

Citations in the corpus

Semantic Segmentation on Swiss3DCities: A Benchmark Study on Aerial Photogrammetric 3D Pointcloud Dataset (2020, methodology): “Semantic3D dataset[12] is a large-scale pointcloud dataset with per-point semantic labels.”
The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes (2019, background): “Net [13] and Oakland dataset [16] are two point cloud datasets that provide semantic labels for point cloud classification.”
Deep Learning for 3D Point Clouds: A Survey (2019, methodology): “Convolution TABLE 4: Comparative semantic segmentation results on the S3DIS (including both Area5 and 6-fold cross validation) [176], Semantic3D (including both semantic-8 and reduced-8 subsets) [9] , ScanNet [8], and SemanticKITTI [177] datasets.”
KPConv: Flexible and Deformable Convolution for Point Clouds (2019, background): “An illustration of segmented scenes on Semantic3D and S3DIS is shown in Figure 4.”
SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks (2017, methodology): “We mainly experiment on the Semantic 3D dataset [40] (http://semantic3d.”
Semantic Stereo for Incidental Satellite Images (2018, methodology): “h a semantic label. This enables development and evaluation of point cloud semantic segmentation algorithms. We borrow the conventions for both file format and scoring methodology from SEMANTIC3D.NET [11]. Performance is assessed using mIoU and overall accuracy (OA). 5. Baseline Methods To validate ”
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review (2020, background): “Semantic3D [56] ASCII X, Y, Z, Intensity, R, G, B 4 billion points 8 Dense training & testing; competing with the most algorithms”
Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene (2020, background): “cially the deep learning. Large and well-annotated datasets play a leading role for the successful application of these techniques. Although dozens of 3D scene-based point cloud datasets are proposed [1, 3, 9, 10, 14, 25, 26, 28, 31], majority of them are not perfectly fit for outdoor scene reconstr”
Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion (2020, neutral): “Therefore, despite the promising performance on complete data (e.g., 80% mIOU on Semantic3D), the semantic segmentation of sparse single sweep LiDAR point cloud still remains a big challenge, which extremely limits its accuracy in real applications.”
Kimera: From SLAM to spatial perception with 3D dynamic scene graphs (2021, methodology): “, 2015), research in these two fields has traditionally proceeded in isolation, and there has been a recent and growing research at the intersection of these areas (Bao and Savarese, 2011; Bowman et al., 2017; Cadena et al., 2016; Davison, 2018; Grinvald et al., 2019; Hackel et al., 2017; Zheng et a”
PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation (2020, methodology): “Other point cloud segmentation datasets, such as Semantic3D [11], are out of the scope of online LiDAR segmentation.”
PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding (2018, background): “Many cutting-edge learning algorithms, especially for 3D understanding [45, 44, 30, 8], intuitive physics [27], and reinforcement learning [48, 29], require such data to train the networks and benchmark the performances.”
FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds (2017, background): “…detection [20], layout inference [16], scene labeling [29], category discovery [57], point classiﬁcation, dense labeling and segmentation [3,10,13,19,21,24,26,35,39,51,52,55], Most deep neural networks designed for 3D point clouds are based on the idea of partitioning the 3D space into regular…”
ShellNet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics (2019, methodology): “Qualitative comparisons between our prediction and the ground truth are shown Method ShapeNet ScanNet S3DIS Semantic3D mpIoU OA mIoU mIoU SyncCNN [45] 82.0 - - - SpiderCNN [42] 81.7 - - - SplatNet [31] 83.7 - - - SO-Net [19] 81.0 - - - SGPN [37] 82.8 - 50.4 - PCNN [3] 81.8 - - - KCNet [30] 82.2 - - ”