Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling

By Yiru Shen, Chen Feng, Yaoqing Yang, Dong Tian. 2017

532 citations recorded by the source corpus.

Unlike on images, semantic learning on 3D point clouds using a deep network is challenging due to the naturally unordered data structure. Among existing works, PointNet has achieved promising results by directly learning on point sets. However, it does not take full advantage of a point's local neighborhood that contains fine-grained structural information which turns out to be helpful towards better semantic learning. In this regard, we present two new operations to improve PointNet with a more efficient exploitation of local structures. The first one focuses on local 3D geometric structures. In analogy to a convolution kernel for images, we define a point-set kernel as a set of learnable 3D points that jointly respond to a set of neighboring data points according to their geometric affinities measured by kernel correlation, adapted from a similar technique for point cloud registration. The second one exploits local high-dimensional feature structures by recursive feature aggregation on a nearest-neighbor-graph computed from 3D positions. Experiments show that our network can efficiently capture local information and robustly achieve better performances on major datasets. Our code is available at http://www.merl.com/research/license#KCNet

Subfields: Computational Mechanics, Environmental Engineering, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Environmental Engineering

In Geology

Citations in the corpus

Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, neutral): “4 w/ normal) KCNet [33] 91 3DmFV [2] 86.”
PointCNN: Convolution On X-Transformed Points (2018, methodology): “Kernel correlation and graph pooling are proposed for improving PointNet-like methods in [42].”
LPD-Net: 3D Point Cloud Learning for Large-Scale Place Recognition and Environment Analysis (2018, methodology): “Inspired by PointNet, different networks have been proposed [13, 23, 17, 5] and achieved advanced point cloud classification and segmentation rear X iv :1 81 2.”
Morphing and Sampling Network for Dense Point Cloud Completion (2019, background): “PointNet (Qi et al. 2017a) uses symmetric functions to aggregate information from individual points, followed by some improvements on local feature learning (Qi et al. 2017b; Shen et al. 2018).”
Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis (2021, unknown): “Our CurveNet outperforms [31, 27] in all experiments and achieves on par results to [44], demonstrating the robustness to noise.”
Patch-Based Progressive 3D Point Set Upsampling (2018, methodology): “Inspired by the dynamic graph convolution [46, 52], we define our local neighborhood in feature space.”
PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks With Adaptive Sampling (2020, methodology): “The comparisons with PointNet [24], PointConv [44] and KC-Net [28] are shown in Fig.”
FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation (2017, methodology): “The graph-based encoder follows a similar design in [46] which focuses on supervised learning using point cloud neighborhood graphs.”
MeshNet: Mesh Neural Network for 3D Shape Representation (2018, methodology): “KCNet learns kernels representing different spatial distributions of point sets, and measures the geometric afﬁnities between kernels and neighboring points for each point to indicate the local structure.”
KPConv: Flexible and Deformable Convolution for Point Clouds (2019, methodology): “Despite the resemblance of vocabulary, our work differs from [31], which is inspired from point cloud registration techniques, and uses kernel points without any weights to learns local geometric patterns.”
Monte Carlo convolution for learning on non-uniformly sampled point clouds (2018, methodology): “is not touched upon. Most of the existing methods are based on convolving the k nearest neighbors (using a hierarchical structure or not) [Groh et al. 2018; Klokov and Lempitsky 2017; Li et al. 2018; Shen et al. 2017; Wang et al. 2018; Xu et al. 2018]. This approach is not robust in non-uniform samp”
ShellNet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics (2019, background): “[30] also improve a PointNet-like network by kernel correlation and graph pooling.”
Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network (2018, background): “KC-Net (Shen et al. 2018) employs a kernel correlation layer and a graph-based pooling layer to capture the local information of point clouds.”
ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning (2020, methodology): “Enhancements to the PointNet architecture include incorporating locality information with kernel correlation [42], and contextual information with LSTMs [30].”
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds (2019, background): “These modules can be generally categorized as: 1) neighbouring feature pooling [44, 32, 21, 70, 69], 2) graph message passing [57, 48, 55, 56, 5, 22, 34], 3) kernel-based convolution [49, 20, 60, 29, 23, 24, 54, 38], and 4) attentionbased aggregation [61, 68, 66, 42].”
DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing (2019, methodology): “The comparisons with PointNet, PointNet++ and KCNet are shown in Fig.”