Shape-based recognition of 3D point clouds in urban environments

By Aleksey Golovinskiy, Vladimir G. Kim, Thomas A. Funkhouser. 2009

479 citations recorded by the source corpus.

This paper investigates the design of a system for recognizing objects in 3D point clouds of urban environments. The system is decomposed into four steps: locating, segmenting, characterizing, and classifying clusters of 3D points. Specifically, we first cluster nearby points to form a set of potential object locations (with hierarchical clustering). Then, we segment points near those locations into foreground and background sets (with a graph-cut algorithm). Next, we build a feature vector for each point cluster (based on both its shape and its context). Finally, we label the feature vectors using a classifier trained on a set of manually labeled objects. The paper presents several alternative methods for each step. We quantitatively evaluate the system and tradeoffs of different alternatives in a truthed part of a scan of Ottawa that contains approximately 100 million points and 1000 objects of interest. Then, we use this truth data as a training set to recognize objects amidst approximately 1 billion points of the remainder of the Ottawa scan.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

DIMENSIONALITY BASED SCALE SELECTION IN 3D LIDAR POINT CLOUDS (2012, background): “More advanced features based on harmonics or spin images (Frome et al., 2004; Golovinskiy et al., 2009) are not necessary since the segmentation task, which indeed requires contextual knowledge, is not tackled in this paper.”
Dynamic Graph CNN for Learning on Point Clouds (2018, background): “nd analysis include indoor navigation [Zhu et al. 2017], self-driving vehicles [Liang et al. 2018; Qi et al. 2017a; Wang et al. 2018b], robotics [Rusu et al. 2008b], and shape synthesis and modeling [Golovinskiy et al. 2009; Guerrero et al. 2018]. These modern applications demand high-level processi”
PointGrid: A Deep Network for 3D Shape Understanding (2018, methodology): “After that, people either directly apply machine learn- ing approaches on these features (e.g. k-NN, random forest, SVM [14], JointBoost classifier [23], correspondence analysis [56, 62]) or employ some local and greedy methods such as region growing [52, 20], hierarchical clustering [3, 11, 24], sp”
Tangent Convolutions for Dense Prediction in 3D (2018, methodology): “Pi-oneering methods work on aerial LiDAR data and are based on hand-crafted features with complex classiﬁers on top [6, 7, 15].”
Min-cut based segmentation of point clouds (2009, methodology): “This was used in the system of [6], for example, to request multiple segmentations of an object for more robust feature extraction.”
Contextually guided semantic labeling and search for three-dimensional point clouds (2013, unknown): “Golovinskiy et al. (2009) capture context by designing node features.”
A search-classify approach for cluttered indoor scene understanding (2012, methodology): “In the preprocessing stage, we compute a classifier by learning shape features from a training set.”
VoxNet: A 3D Convolutional Neural Network for real-time object recognition (2015, methodology): “Most of this work uses a pipeline combining various hand-crafted features or descriptors with a machine learning classifier ([10], [11], [12], [13]).”
Semantic Labeling of 3D Point Clouds for Indoor Scenes (2011, unknown): “[8, 30] capture context by designing node features and [36] do so by stacking layers of classifiers; however these methods do not model the correlation between the labels.”
Automatic Reconstruction of As-Built Building Information Models from Laser-Scanned Point Clouds: A Review of Related Techniques | NIST (2010, background): “From early work on detecting objects on a conveyer belt — the “binpicking” task [24] — to more recent work on recognizing vehicles and other objects in urban environments [31,61], a successful strategy to object instance recognition has evolved [15].”
Neighbors Do Help: Deeply Exploiting Local Structures of Point Clouds (2017, methodology): “K nearest neighbor (KNN) graph is usually used to establish local connectivity information, in the applications of point cloud on surface detection, 3D object recognition, 3D object segmentation and compression [5, 25, 27].”
Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling (2017, unknown): “KNNG is usually used to establish local connectivity information, in the applications of point cloud on surface detection, 3D object recognition, 3D object segmentation and compression [12,35,37].”
FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds (2017, methodology): “K-NNGs are also used in other applications of point clouds without the deep learning framework such as surface detection, 3D object recognition, 3D object segmentation and compression [18, 47, 48].”
SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks (2017, background): “In urban classification [2,21] , classifying small objects like cars or street furniture [3] and discriminating between roads and natural terrain become decisive at the smallest possible scale: point level [2] .”
Perception, Planning, Control, and Coordination for Autonomous Vehicles (2017, methodology): “In [59,60], the authors first created a k-nearest neighbors graph, assigned each node according to a background penalty function, added hard foreground constraints, and solved the foreground and background segmentation through min-cut.”