Neighbors Do Help: Deeply Exploiting Local Structures of Point Clouds

By Yiru Shen, Chen Feng, Yaoqing Yang, Dong Tian. 2017

33 citations recorded by the source corpus.

Unlike on images, semantic learning on 3D point clouds using a deep network is challenging due to the naturally unordered data structure. Among existing works, PointNet has achieved promising results by directly learning on point sets. However, it does not take full advantage of a point's local neighborhood that contains fine-grained structural information which turns out to be helpful towards better semantic learning. In this regard, we present two new operations to improve PointNet with more efficient exploitation of local structures. The first one focuses on local 3D geometric structures. In analogy with a convolution kernel for images, we define a point-set kernel as a set of learnable points that jointly respond to a set of neighboring data points according to their geometric affinity measured by kernel correlation, adapted from a similar technique for point cloud registration. The second one exploits local feature structures by recursive feature aggregation on a nearest-neighbor-graph computed from 3D positions. Experiments show that our network is able to robustly capture local information and efficiently achieve better performance on major datasets.

Subfields: Computational Mechanics, Environmental Engineering, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Yue Wang
2. Justin M. Solomon
3. Slobodan Ilic
4. Yue Gao
5. Yifan Feng
6. Rongrong Ji
7. Tolga Birdal
8. Haowen Deng

In Environmental Engineering

1. Justin M. Solomon
2. Yue Wang
3. Slobodan Ilic

In Geology

1. Yue Wang
2. Justin M. Solomon
3. Yue Gao
4. Slobodan Ilic
5. Yifan Feng
6. Haowen Deng
7. Tolga Birdal
8. Rongrong Ji

Citations in the corpus

• Dynamic Graph CNN for Learning on Point Clouds (2018, background): “part segmentation testing results for tables, chairs and lamps. compare our results with PointNet [Qi et al. 2017b], PointNet++ [Qi et al. 2017c], Kd-Net [Klokov and Lempitsky 2017], LocalFeatureNet [Shen et al. 2017], PCNN [Atzmon et al. 2018], and PointCNN [Li et al. 2018a]. The evaluation results”
• PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors (2018, background): “While not being the most powerful network, it clearly sets out a successful architecture giving rise to many successive studies [31, 29, 2, 36].”
• Monte Carlo convolution for learning on non-uniformly sampled point clouds (2018, methodology): “Most of the existing methods are based on convolving the k nearest neighbors (using a hierarchical structure or not) [Groh et al. 2018; Klokov and Lempitsky 2017; Li et al. 2018; Shen et al. 2017; Wang et al. 2018; Xu et al. 2018].”
• PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet (2019, background): “Another variant considers aggregates features of nearby points [34].”
• PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition (2018, background): “Furthermore, subsequent models [24, 25, 31] have proven that considering the neighborhoods of points rather than treating points independently can better leverage the local structure features to improve the performance.”