Point convolutional neural networks by extension operators

By Matan Atzmon, Haggai Maron, Yaron Lipman. 2018

584 citations recorded by the source corpus.

This paper presents Point Convolutional Neural Networks (PCNN): a novel framework for applying convolutional neural networks to point clouds. The framework consists of two operators: extension and restriction, mapping point cloud functions to volumetric functions and vise-versa. A point cloud convolution is defined by pull-back of the Euclidean volumetric convolution via an extension-restriction mechanism. The point cloud convolution is computationally efficient, invariant to the order of points in the point cloud, robust to different samplings and varying densities, and translation invariant, that is the same convolution kernel is used at all points. PCNN generalizes image CNNs and allows readily adapting their architectures to the point cloud setting. Evaluation of PCNN on three central point cloud learning benchmarks convincingly outperform competing point cloud learning methods, and the vast majority of methods working with more informative shape representations such as surfaces and/or normals.

Subfields: Computational Mechanics, Environmental Engineering, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Environmental Engineering

In Geology

Citations in the corpus

Dynamic Graph CNN for Learning on Point Clouds (2018, neutral): “…we have a different graph G ( l ) = (V ( l ) , E ( l ) ) , where the l -th layer edges are of the form ( i , j i 1 ) , . . . , ( i , j ik l ) such that x ( l ) i architecture learns how to construct the graph G used in each layer distance matrix in feature space and then take the closest k points”
PointCNN: Convolution On X-Transformed Points (2018, methodology): “ShapeNet Parts S3DIS ScanNet pIoU mpIoU mIoU OA SyncSpecCNN [55] 84.74 82.0 - - Pd-Network [22] 85.49 82.7 - - SSCN [12] 85.98 83.3 - - SPLATNet [43] 85.4 83.7 - - SpiderCNN [53] 85.3 81.7 - - SO-Net [27] 84.9 81.0 - - PCNN [3] 85.1 81.8 - - KCNet [42] 83.7 82.2 - - SpecGCN [46] 85.4 - - - Kd-Net [2”
PCT: Point cloud transformer (2020, methodology): “Net [16] 84.9 82.8 77.8 88.0 77.3 90.6 73.5 90.7 83.9 82.8 94.8 69.1 94.2 80.9 53.1 72.9 83.0 PointNet++ [22] 85.1 82.4 79.0 87.7 77.3 90.8 71.8 91.0 85.9 83.7 95.3 71.6 94.1 81.3 58.7 76.4 82.6 PCNN [1] 85.1 82.4 80.1 85.5 79.5 90.8 73.2 91.3 86.0 85.0 95.7 73.2 94.8 83.3 51.0 75.0 81.8 DGCNN [29] ”
Patch-Based Progressive 3D Point Set Upsampling (2018, background): “com) with 3D point sets, however, is challenging since, unlike images, the data is unstructured and irregular [3,17,19,34,55].”
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, methodology): “After that, many methods [34, 38, 3, 23, 12, 43, 39, 40, 20, 19, 49, 47, 44, 46, 15] are proposed to perform point cloud classification and segmentation.”
Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review (2020, methodology): “More specifically, the PCNN [33] defines convolution kernels as 3D points with weights.”
Group-Free 3D Object Detection via Transformers (2021, background): “Network architecture for Point Cloud A large set of network architectures [38, 12, 29, 9, 47, 23, 44, 39, 45, 28, 30, 33, 43, 40, 17, 1, 41, 49, 10, 22] have been proposed for various point cloud based learning tasks.”
Lagrangian Fluid Simulation with Continuous Convolutions (2020, methodology): “A number of methods transferred the convolution concept to point clouds in the context of semantic classiﬁcation and segmentation of 3D objects (Hua et al., 2018; Atzmon et al., 2018; Hermosilla et al., 2018; Li et al., 2018; Su et al., 2018; Wu et al., 2019; Xu et al., 2018; Lei et al., 2019).”
PRNet: Self-Supervised Learning for Partial-to-Partial Registration (2019, methodology): “Follow-up works incorporate local information, including PointNet++ [21], DGCNN [22], PointCNN [23], and PCNN [24].”
OpenMask3D: Open-Vocabulary 3D Instance Segmentation (2023, unknown): “Given a 3D scene as input, 3D semantic segmentation task aims to assign a semantic category to each point in the scene [2–4, 9, 14, 15, 22, 28, 29, 31, 38, 42, 44, 46, 47, 53, 54, 63, 64, 66, 68, 70, 73].”
PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks With Adaptive Sampling (2020, background): “The underlying reason is that, currently, most appealing methods for feature learning is to query a local group around the each sampled point, and then they construct the graph-based learning [30, 42, 50, 14] or define convolution-like operations [12, 47, 8, 3, 44, 34] (we denote them as Point Local”
Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline (2021, methodology): “…of DNNs have emerged to process 3D points directly (Simonovsky & Komodakis, 2017; Zaheer et al., 2017; Klokov & Lempitsky, 2017; Xu et al., 2018; Atzmon et al., 2018; Wang et al., 2018a; Li et al., 2018a; Groh et al., 2018; Ben-Shabat et al., 2018; Xie et al., 2018; Li et al., 2018b; Liu et…”
Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds (2020, background): “To tackle this problem, current state-of-the-arts on point cloud analysis [2,26,28,33,38,43,49,51,54] are established with the help of extensive human-annotated supervised information.”
DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing (2019, methodology): “Random input dropout is applied during training, for fair comparisons with PointNet [32], PointNet++ [34], SO-Net [24], PCNN [1] and DGCNN [52].”
Deep Closest Point: Learning Representations for Point Cloud Registration (2019, methodology): “PCNN [3] uses an extension operator to define convolution on point clouds, while PointCNN [25] applies Euclidean convolution after applying a learned transformation.”
MeshCNN: a network with an edge (2019, methodology): “Subsequently, feature computation is performed on top of the edges themselves, while leveraging mesh decimation techniques, such as edge collapse, in a manner that adheres to shape geometry and topology.”
Self-Supervised Pretraining of 3D Features on any Point-Cloud (2021, methodology): “Point-cloud based models [66, 68] are widely used in classification and segmentation tasks [6, 40, 45, 66, 68, 91, 98, 100, 106, 107], 3D reconstruction [23, 90, 110] and 3D object detection [63, 64, 77, 98, 112, 112, 114].”