Point Transformer
By Nico Engel, Vasileios Belagiannis, Klaus C. J. Dietmayer. 2020
3,096 citations recorded by the source corpus.
In this work, we present Point Transformer, a deep neural network that operates directly on unordered and unstructured point sets. We design Point Transformer to extract local and global features and relate both representations by introducing the local-global attention mechanism, which aims to capture spatial point relations and shape information. For that purpose, we propose SortNet, as part of the Point Transformer, which induces input permutation invariance by selecting points based on a learned score. The output of Point Transformer is a sorted and permutation invariant feature list that can directly be incorporated into common computer vision applications. We evaluate our approach on standard classification and part segmentation benchmarks to demonstrate competitive results compared to the prior work.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jiashi Feng
- Jitendra Malik
- Yi Yang
In Geology
Citations in the corpus
- Transformers in Vision: A Survey (2021, methodology): “However, compared to [173], it is more directly based on the conventional Transformer architecture [1] and does not involve vector attention.”
- Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet (2021, background): “Recent works have explored to apply Transformers to various vision tasks: image classification [5, 12], object detection [3, 59, 56, 8, 34], segmentation [4, 40], image enhancement [4, 45], image generation [27], video processing [58, 51], and 3D point cloud processing [54].”
- CvT: Introducing Convolutions to Vision Transformers (2021, background): “…a viable alternative to the convolutional neural networks (CNNs) in visual recognition tasks, such as classification [11, 30], object detection [3, 45, 43, 8, 28], segmentation [33, 36], image enhancement [4, 40], image generation [24], video processing [42, 44] and 3D point cloud processing [12].”
- ViViT: A Video Vision Transformer (2021, background): “ViT has inspired a large amount of follow-up work in the community, and we note that there are a number of concurrent approaches on extending it to other tasks in computer vision [71, 74, 84, 85] and improving its data-efficiency [64, 48].”
- FlowFormer: A Transformer Architecture for Optical Flow (2022, background): “Since then, transformer-based architectures has been introduced into many other vision tasks, such as detection [4], point cloud processing [15,59], image restoration [6,31], video inpainting[57,32], etc, and achieves state-of-the-art in most tasks.”
- PointMamba: A Simple State Space Model for Point Cloud Analysis (2024, background): “On the other hand, some researchers [20, 65, 51] focus on modifying the Transformers for point clouds.”
- Skeleton-based action recognition via spatial and temporal transformer networks (2020, background): “Zhao et al. (2020) proposed Point Transformer, a model which uses transformer self-attention to encode relations between point clouds, exploiting their permutation invariant nature.”
- Transformer-Based Visual Segmentation: A Survey (2023, background): “The works [29], [255] focus on transferring the success in image/video representation learning into the point cloud.”
- Attention mechanisms in computer vision: A survey (2021, background): “In addition to providing significant improvements in the image domain, it also has been proven useful in 3D point cloud processing [80].”
- TransGAN: Two Transformers Can Make One Strong GAN (2021, background): “Besides image classification task, transformer and its variants are also explored on image processing (Chen et al., 2020a), point cloud (Zhao et al., 2020a), object detection (Carion et al., 2020; Zhu et al., 2020) and so on.”
- Voxel Transformer for 3D Object Detection (2021, background): “Point Trans-former [41] designs a novel point operator for point cloud classification and segmentation.”
- UniT: Multimodal Multitask Learning with a Unified Transformer (2021, neutral): “[61, 47, 11, 69, 70]).”