PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation

By Mingyang Jiang, Yiran Wu, Zelin Zhao, Cewu Lu. arXiv.org, 2018

439 citations recorded by the source corpus.

Recently, 3D understanding research sheds light on extracting features from point cloud directly, which requires effective shape pattern description of point clouds. Inspired by the outstanding 2D shape descriptor SIFT, we design a module called PointSIFT that encodes information of different orientations and is adaptive to scale of shape. Specifically, an orientation-encoding unit is designed to describe eight crucial orientations, and multi-scale representation is achieved by stacking several orientation-encoding units. PointSIFT module can be integrated into various PointNet-based architecture to improve the representation ability. Extensive experiments show our PointSIFT-based framework outperforms state-of-the-art method on standard benchmark datasets. The code and trained model will be published accompanied by this paper.

Subfields: Computational Mechanics, Environmental Engineering, Geology

View Paper: https://www.semanticscholar.org/paper/ac7072f8446cafabfade74e274ca4b2a90587e61

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Daniel Cohen-Or
2. Olga Sorkine-Hornung
3. Shaoshuai Shi
4. Jiaya Jia
5. Hongsheng Li
6. Shihao Wu
7. Xiaogang Wang
8. Yifan Wang

In Environmental Engineering

1. Shaoshuai Shi
2. Cyrill Stachniss
3. Jens Behley
4. Andres Milioto
5. Xiaogang Wang
6. Hongsheng Li
7. Juergen Gall
8. Sven Behnke

In Geology

1. Shaoshuai Shi
2. Jiaya Jia
3. Xiaogang Wang
4. Hongsheng Li
5. Shu Liu
6. Zetong Yang
7. Yanan Sun
8. Yue Gao

Citations in the corpus

• GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, methodology): “After that, many methods [34, 38, 3, 23, 12, 43, 39, 40, 20, 19, 49, 47, 44, 46, 15] are proposed to perform point cloud classification and segmentation.”
• Relation-Shape Convolutional Neural Network for Point Cloud Analysis (2019, background): “, image) analysis [17, 32], to irregular point cloud processing [26, 15, 45, 29, 27, 34, 38].”
• IPOD: Intensive Point-based Object Detector for Point Cloud (2018, neutral): “Other solutions, such as [31, 30, 22, 17, 21], segment the point cloud from raw LI-DAR data.”
• Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution (2018, methodology): “ture extraction stage to perform better on various tasks. PCNN[1] utilizes extension and restriction operators to transform point clouds to Euclidean volumetric space for better performance. PointSIFT[8] proposes an innovative SIFT-like feature learning method, which is more robust in semantic segme”
• Patch-Based Progressive 3D Point Set Upsampling (2018, background): “A few recent works have exploited the power of multiscale representation [12, 24, 28, 37, 49] and skipconnection [8,43] in 3D learning.”
• SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences (2019, methodology): “To overcome the limitations of the voxel-based representation, such as the exploding memory consumption when the resolution of the voxel grid increases, more recent approaches either upsample voxel-predictions [53] using a CRF or use different representations, like more efficient spatial subdivision”
• Morphing and Sampling Network for Dense Point Cloud Completion (2019, background): “There are also lots of works exploring specialized convolution operation for point clouds (Jiang, Wu, and Lu 2018; Li et al. 2018; Liu et al. 2019).”
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud (2019, background): “Other solutions, such as [28, 27, 18, 12, 17], segment point clouds from raw LiDAR data.”
• MeshNet: Mesh Neural Network for 3D Shape Representation (2018, background): “Self-organizing network (SO-Net) (Li, Chen, and Lee 2018), kernel correlation network (KCNet) (Shen et al. ) and PointSIFT (Jiang, Wu, and Lu 2018) develop more detailed approaches for capturing local structures with nearest neighbors.”
• Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review (2020, background): “To achieve scale-invariant, the PointSIFT [64] is incorporated into the network.”
• PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud (2018, methodology): “ures for describing the raw point clouds, we utilize the PointNet++ [25] with multi-scale grouping as our backbone network. There are several other alternative point-cloud network structures, such as [23,12], which could also be adopted as our backbone network. Foreground point segmentation. The for”