PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
By Charles Ruizhongtai Qi, Li Yi, Hao Su, Leonidas J. Guibas. Neural Information Processing Systems, 2017
14,555 citations recorded by the source corpus.
Few prior works study deep learning on point sets. PointNet by Qi et al. is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.
Subfields: Aerospace Engineering, Computational Mechanics, Geology, 3D Vision & Pose
View Paper: https://arxiv.org/abs/1706.02413
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Andreas Geiger
- Jitendra Malik
- Leonidas J. Guibas
- Vladlen Koltun
- Matthias Nießner
- Marc Pollefeys
- Xiaogang Wang
- Charles Ruizhongtai Qi
In Computational Mechanics
- Andreas Geiger
- Leonidas J. Guibas
- Matthias Nießner
- Charles Ruizhongtai Qi
- Michael Niemeyer
- Or Litany
- Yaron Lipman
- Thomas A. Funkhouser
In Geology
- Leonidas J. Guibas
- Charles Ruizhongtai Qi
- Matthias Nießner
- Binh-Son Hua
- Sai-Kit Yeung
- Or Litany
- Angela Dai
- Jiwen Lu
In 3D Vision & Pose
- Andreas Geiger
- Jitendra Malik
- Leonidas J. Guibas
- Michael Niemeyer
- Matthias Nießner
- Gerard Pons-Moll
- Yaron Lipman
- Justin Johnson
Citations in the corpus
- 3D Instance Segmentation of MVS Buildings (2021, background): “Through point grouping and multi-level feature extraction, PointNet++ can better extract discriminative features for point clouds with uneven density [6].”
- Fast Graph Representation Learning with PyTorch Geometric (2019, methodology): “…manifolds and graphs with multi-dimensional edge features, we provide the relational GCN operator from Schlichtkrull et al. (2018), PointNet++ (Qi et al., 2017), PointCNN (Li et al., 2018), and the continuous kernel-based methods MPNN (Gilmer et al., 2017), MoNet (Monti et al., 2017),…”
- VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection (2017, background): “tures as in [31, 32] results in high computational and memory requirements.”
- Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, background): “Qi et al. [54, 56] pioneered point clouds as a representation for discriminative deep learning tasks.”
- Flexible Neural Representation for Physics Prediction (2018, background): “Representations using 3D convolution or point clouds are better at maintaining object shape [46, 47, 10, 36, 37], but do not entirely capture object permanence, and can be computationally inefficient.”
- SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020, background): “DeepSet[30] 3DmFV[3] Set Transformer[12] PointNet[15] SpiderCNN[29] Tensor Field +z[20] PointNet++[16] SE(3)-Transf.”
- Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, methodology): “It can be explained by the fact that (c) SpiderCNN [44], (d) PointNet++ [32], (e) DGCNN [42] and (f) PointCNN [25] on our hardest PB T50 RS.”
- Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds (2018, background): “The networks presented in Qi et al. [11, 12] operate on point clouds and use symmetric functions to encode permutation invariance, but these networks do not include rotation equivariance.”
- Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, positive): “seeds) from N input points with enriched features via PointNet++ (Qi et al. 2017); 2) voting module that generates expected centers of objects (i.”
- Transformers in Vision: A Survey (2021, background): “a) Only moderate improvements over previous state-of-the-art, b) Large number of trainable parameters around 6× higher than PointNet++ [156].”
- Dynamic Graph CNN for Learning on Point Clouds (2018, methodology): “] 83.0 85.9 Subvolume [Qi et al. 2016] 86.0 89.2 VRN (single view) [Brock et al. 2016] 88.98 - VRN (multiple views) [Brock et al. 2016] 91.33 - ECC [Simonovsky and Komodakis 2017] 83.2 87.4 PointNet [Qi et al. 2017b] 86.0 89.2 PointNet++ [Qi et al. 2017c] - 90.7 Kd-net [Klokov and Lempitsky 2017] - ”
- DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation (2019, methodology): “PointNet [38, 39], for example, uses max-pool operations to extract global shape features, and the technique is widely used as an encoder for point generation networks [57, 1].”
- Learning Implicit Fields for Generative Shape Modeling (2018, background): “There have been a variety of 3D shape representations for deep learning of shapes, such as voxel grids [9, 14, 28, 45, 46], octrees [18, 35, 40, 43, 44], multi-view images [29, 39], point clouds [1, 12, 13, 32, 33, 47, 48], geometry images [37, 38], deformable mesh/patches [16, 38, 42, 47], and part”
- A Papier-Mache Approach to Learning 3D Surface Generation (2018, neutral): “In contrast, a point cloud [24, 25] provides a representation for generating on-surface details [8], efficiently leveraging sparsity of the data.”
- Learning Data Augmentation Strategies for Object Detection (2019, background): “Likewise, point cloud featurizations [35, 36] are another domain that has a rich set of possibilities for geometric data augmentation operations, and can benefit from an approach similar to the one taken here.”
- Neural 3D Mesh Renderer (2017, methodology): “Point clouds have been used for both recognition [12, 19, 21] and reconstruction [5].”