Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

By Xinhai Liu, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker. 2018

359 citations recorded by the source corpus.

Exploring contextual information in the local region is important for shape understanding and analysis. Existing studies often employ hand-crafted or explicit ways to encode contextual information of local regions. However, it is hard to capture fine-grained contextual information in hand-crafted or explicit manners, such as the correlation between different areas in a local region, which limits the discriminative ability of learned features. To resolve this issue, we propose a novel deep learning model for 3D point clouds, named Point2Sequence, to learn 3D shape features by capturing fine-grained contextual information in a novel implicit way. Point2Sequence employs a novel sequence learning model for point clouds to capture the correlations by aggregating multi-scale areas of each local region with attention. Specifically, Point2Sequence first learns the feature of each area scale in a local region. Then, it captures the correlation between area scales in the process of aggregating all area scales using a recurrent neural network (RNN) based encoder-decoder structure, where an attention mechanism is proposed to highlight the importance of different area scales. Experimental results show that Point2Sequence achieves state-of-the-art performance in shape classification and segmentation tasks.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, methodology): “In the online leaderboard maintained by the authors of ModelNet40, the accuracy of the object classiﬁcation task is reaching perfection, with 92% for point cloud methods [25, 42, 44, 26].”
Point Transformer (2020, methodology): “We compare our results with approaches that operate directly on 3D point sets [7], [13], [14], attentionbased approaches [17], [20], [35] and methods that use point cloud convolutions [29]–[31], [39].”
Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis (2021, methodology): “We split the training and testing samples following the same schema as in [19].”
PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks With Adaptive Sampling (2020, methodology): “Point2Sequence [21] uses the attention mechanism to aggregate the information of different local regions.”
ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning (2020, methodology): “Enhancements to the PointNet architecture include incorporating locality information with kernel correlation [42], and contextual information with LSTMs [30].”
Geometric Back-Projection Network for Point Cloud Classification (2019, background): “s [50,10,18] derive benets from both spatial and channel domains of 2D images. In terms of 3D point clouds, attention modules contribute to point clouds detection [31], generation [40], segmentation [55,20,56], etc. However, limited work is done in well-designed attention mechanisms targeting 3D poi”
Revealing Occlusions with 4D Neural Fields (2022, background): “More recently, researchers have begun to apply transformer attention mechanisms that were ﬁrst found to be valuable in the language domain [58] to encode point clouds [27,28,64].”
SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer (2021, background): “The development of deep learning based 3D point cloud processing techniques [10, 52, 49, 35, 33, 34, 32, 21, 3, 36] have boosted the research of point cloud completion.”
KPConv: Flexible and Deformable Convolution for Point Clouds (2019, methodology): “Following PointNet, some hierarchical architectures have been developed to aggregate local neighborhood information with MLPs [26, 18, 20].”
Stratified Transformer for 3D Point Cloud Segmentation (2022, neutral): “…82.4 85.3 SPLATNet [37] 83.7 85.4 DGCNN [50] 82.3 85.2 SubSparseCNN [15] 83.3 86.0 PointCNN [22] 84.6 86.1 PointConv [52] 82.8 85.7 Point2Sequence [23] - 85.2 PVCNN [27] - 86.2 RS-CNN [25] 84.0 86.2 KPConv [41] 85.0 86.2 InterpCNN [30] 84.0 86.3 DensePoint [24] 84.2 86.4 PAConv [54] 84 Point…”
Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution (2018, methodology): “Point2Sequence (Liu et al. 2019) uses a recurrent neural network (RNN) based encoder-decoder structure, where an attention mechanism is proposed to highlight the importance of different area scales.”
Patch-Based Progressive 3D Point Set Upsampling (2018, background): “A few recent works have exploited the power of multiscale representation [12, 24, 28, 37, 49] and skipconnection [8,43] in 3D learning.”
ShellNet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics (2019, methodology): “[22] use a sequence model to summarize local features with multiple scales.”