PointCNN: Convolution On X-Transformed Points
By Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, Baoquan Chen. 2018
2,957 citations recorded by the source corpus.
We present a simple and general framework for feature learning from point clouds. The key to the success of CNNs is the convolution operator that is capable of leveraging spatially-local correlation in data represented densely in grids (e.g. images). However, point clouds are irregular and unordered, thus directly convolving kernels against features associated with the points, will result in desertion of shape information and variance to point ordering. To address these problems, we propose to learn an $\mathcal{X}$-transformation from the input points, to simultaneously promote two causes. The first is the weighting of the input features associated with the points, and the second is the permutation of the points into a latent and potentially canonical order. Element-wise product and sum operations of the typical convolution operator are subsequently applied on the $\mathcal{X}$-transformed features. The proposed method is a generalization of typical CNNs to feature learning from point clouds, thus we call it PointCNN. Experiments show that PointCNN achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.
Subfields: Aerospace Engineering, Computational Mechanics, Computer Vision and Pattern Recognition, Geology, Model Architectures, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Hongsheng Li
- Leonidas J. Guibas
- Shaoshuai Shi
- Vladlen Koltun
- Or Litany
- Sai-Kit Yeung
- Binh-Son Hua
- Charles Ruizhongtai Qi
In Computational Mechanics
- Leonidas J. Guibas
- Or Litany
- Rohit Girdhar
- Ishan Misra
- Armand Joulin
- Charles Ruizhongtai Qi
- Matthias Nießner
- Chi-Wing Fu
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Jitendra Malik
- Trevor Darrell
- Yi Yang
- Andrea Vedaldi
In Geology
- Or Litany
- Shaoshuai Shi
- Li Jiang
- Jiaya Jia
- Hongsheng Li
- Leonidas J. Guibas
- Charles Ruizhongtai Qi
- Rohit Girdhar
In Model Architectures
- Xiaogang Wang
- Jitendra Malik
- Chen Change Loy
- Ming-Ming Cheng
- Hongsheng Li
- Yu Qiao
- Dacheng Tao
- Jiaya Jia
In 3D Vision & Pose
- Leonidas J. Guibas
- Charles Ruizhongtai Qi
- Or Litany
- Sai-Kit Yeung
- Binh-Son Hua
- Michael M. Bronstein
- Rohit Girdhar
- M. Ovsjanikov
Citations in the corpus
- Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, methodology): “We re-trained PointNet [30], PointNet++ [32], PointCNN [25], Dynamic Graph CNN (DGCNN) [42], 3D modified Fisher Vector (3DmFV) [2], and SpiderCNN [44].”
- Dynamic Graph CNN for Learning on Point Clouds (2018, neutral): “2017] and PointCNN [Li et al. 2018a].”
- Distilling Knowledge From Graph Convolutional Networks (2020, methodology): “Another dataset is ModelNet40 [40] that contains point clouds come from the CAD models, which is a common used dataset for 3D object analysis [28, 30, 19, 37].”
- Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks (2021, neutral): “7 - - - - - - - - - - PointCNN [91] 86.”
- Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023, background): “The similar idea has been explored in term of continuous 3D point cloud convolution [49–51].”
- PointCLIP: Point Cloud Understanding by CLIP (2021, methodology): “With 3D sensing technology developing rapidly, the growing demand for processing 3D point cloud data has boosted many advanced deep models with better local feature ag-gregator [30, 32, 49], geometry modeling [20, 39, 40] and projection-based processing [20, 34, 48].”
- Lagrangian Fluid Simulation with Continuous Convolutions (2020, methodology): “A number of methods transferred the convolution concept to point clouds in the context of semantic classification and segmentation of 3D objects (Hua et al., 2018; Atzmon et al., 2018; Hermosilla et al., 2018; Li et al., 2018; Su et al., 2018; Wu et al., 2019; Xu et al., 2018; Lei et al., 2019).”
- PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies (2022, positive): “1 %), outperforming many modern architectures that are subsequent to PointNet++, such as PointCNN [21] and DeepGCN [20].”
- I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting (2020, methodology): “After initializing the locations of L structural centroids via the farthest point sampling (Qi et al. 2017b; Li et al. 2018) over the point cloud, we collect the corresponding k nearest neighbor points around each structural centroid to construct L geometric structures.”
- Point Transformer V2: Grouped Vector Attention and Partition-based Pooling (2022, methodology): “Meanwhile, our model is still lightweight compared to voxel-based backbones, such as MinkUNet42 [18] with 37.9M parameters.”
- Multi-directional geodesic neural networks via equivariant convolution (2018, methodology): “In their recent work [Li et al. 2018] Li et al. applied PointCNN to the CIFAR-10 classification benchmark.”
- PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds (2021, background): “PointCNN [24] learns an X -transformation to relate points with kernels.”
- PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, background): “Recently representation learning on point clouds has drawn lots of attention for improving the performance of point cloud classification and segmentation [10], [18], [19], [68], [69], [70], [71], [72], [73], [74], [75], [76], [77], [78], [79].”
- PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection (2019, methodology): “ng. Representation Learning on Point Clouds. Recently representation learning on point clouds has drawn lots of attention for improving the performance of point cloud classifi- cation and segmentation [23, 24, 41, 31, 7, 38, 15, 28, 33, 8, 29, 3]. In terms of 3D detection, previous methods generally ”
- KPConv: Flexible and Deformable Convolution for Point Clouds (2019, methodology): “As shown by [40, 19, 13], the kernel of a point convolution can be implemented with a MLP, because of its ability to approximate any continuous function.”