Point Transformer V2: Grouped Vector Attention and Partition-based Pooling

By Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, Hengshuang Zhao. Neural Information Processing Systems, 2022

828 citations recorded by the source corpus.

As a pioneering work exploring transformer architecture for 3D point cloud understanding, Point Transformer achieves impressive results on multiple highly competitive benchmarks. In this work, we analyze the limitations of the Point Transformer and propose our powerful and efficient Point Transformer V2 model with novel designs that overcome the limitations of previous work. In particular, we first propose group vector attention, which is more effective than the previous version of vector attention. Inheriting the advantages of both learnable weight encoding and multi-head attention, we present a highly effective implementation of grouped vector attention with a novel grouped weight encoding layer. We also strengthen the position information for attention by an additional position encoding multiplier. Furthermore, we design novel and lightweight partition-based pooling methods which enable better spatial alignment and more efficient sampling. Extensive experiments show that our model achieves better performance than its predecessor and achieves state-of-the-art on several challenging 3D point cloud understanding benchmarks, including 3D point cloud segmentation on ScanNet v2 and S3DIS and 3D point cloud classification on ModelNet40. Our code will be available at https://github.com/Gofinge/PointTransformerV2.

Subfields: Computational Mechanics, Environmental Engineering, Geology

View Paper: https://www.semanticscholar.org/paper/71a1503732902d925993e09d8af9f209932ba978

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Xiaoyang Wu
2. Xihui Liu
3. Zhikang Zou
4. Xiang Bai
5. Xin Zhou
6. Xiao Tan
7. Xingkui Zhu
8. Xiaoqing Ye

In Environmental Engineering

1. Xiaoyang Wu
2. Xihui Liu
3. Hengshuang Zhao
4. Zhijian Liu

In Geology

1. Xiaoyang Wu
2. Xihui Liu
3. Hengshuang Zhao
4. Zhijian Liu

Citations in the corpus

• PointMamba: A Simple State Space Model for Point Cloud Analysis (2024, background): “The later Point Transformer series [55, 54] further extends the performance and efficiency of the Transformer for different tasks.”
• Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, neutral): “Recently, to achieve model scalability, Transformers-based (e.g., PointTransformer [78], Swin3d [79], PT2 [80], 3D-VisTA [81], LEO [82], PQ3D [83]) and Mamba-based (e.g., PointMamba [84], Mamba3D [85]) architectures have emerged.”
• Point Transformer V3: Simpler, Faster, Stronger (2023, methodology): “Coupled with this strategy, the efficient convolutional backbone [13] has effectively bridged the accuracy gap commonly associated with point cloud transformers [40, 90].”