PCT: Point cloud transformer
By Meng-Hao Guo, Junxiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph Robert Martin, Shimin Hu. 2020
2,195 citations recorded by the source corpus.
The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer (PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation, semantic segmentation, and normal estimation tasks.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jiashi Feng
- Jitendra Malik
- Bernt Schiele
In Geology
In Model Architectures
- Christoph Feichtenhofer
- Dacheng Tao
- Jiashi Feng
- Yu Qiao
- Hongsheng Li
- Jitendra Malik
- Yanghao Li
- Fahad Shahbaz Khan
Citations in the corpus
- Perceiver: General Perception with Iterative Attention (2021, background): “Because of this state of affairs, best-practice architectures for vision cannot be ported to all domains, and specialized models have been developed to handle domains like point clouds (Qi et al., 2017; Guo et al., 2020).”
- Transformers in Vision: A Survey (2021, background): “The Point Cloud Transformer (PCT) [174] is a parallel work to [173] and motivated by the permutation invariance property of Transformers.”
- Multiscale Vision Transformers (2021, background): “An emerging thread of work aims at applying transformers to vision tasks such as object detection [5], semantic segmentation [121, 105], 3D reconstruction [77], pose estimation [113], generative modeling [17], image retrieval [30], medical image segmentation [16,103,117], point clouds [45], video in”
- FaceFormer: Speech-Driven 3D Facial Animation with Transformers (2021, methodology): “In computer graphics, transformers have been exploited for 3D point cloud representations and 3D mesh, such as Point Transformer [68], Point Cloud Transformer [26] and Mesh Transformer [38].”
- A Survey on Vision--Language--Action Models for Embodied AI (2024, neutral): “Real : SayCan setup LEO [159] ConvNeXt [160], PointNet++ [161] Vicuna-7B Concat MLP Yes Sim-Mani : CLIPort task subset; Sim-Navi : Habitat-web 3D-LLM [57] CLIP, 3D-CLR [162], ConceptFusion [163] Flamingo, BLIP-2”
- Attention mechanisms in computer vision: A survey (2021, background): “Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
- Improving 3D Object Detection with Channel-wise Transformer (2021, methodology): “For point clouds, recent methods [9, 6] also explored to use selfattention for classification and segmentation tasks.”
- Masked Autoencoders for Point Cloud Self-supervised Learning (2022, background): “…Point-MAE 93.8% Supervised methods Accuracy PointNet [29] 89.2% PointNet++ [30] 90.7% PointCNN [21] 92.5% KPConv [38] 92.9% DGCNN [21] 92.9% RS-CNN [24] 92.9% [T]PCT [16] 93.2% [T]PVT [57] 93.6% [T]PointTransformer [59] 93.7% [ST]Transformer [54] 91.4% as ModelNet40 is a relatively small dataset.”
- Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling (2021, methodology): “Although we mainly focus on pre-training for standard Transformers in this work, our MPM pre-training strategy is also suitable for other Transformer-based point cloud models [11, 63].”
- A Survey of Visual Transformers (2021, methodology): “In view of this, Point Transformer [106] and PCT [107] firstly demonstrate the efficacy of the visual Transformer for 3D representation learning.”
- TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions (2021, methodology): “Transformers in low-level vision: Since the introduction of Vision Transformer (ViT) [9] for visual recognition, transformers have been widely adopted for various computer vision tasks [12, 31, 49, 60, 75].”
- A Survey on Vision Transformer (2020, methodology): “A number of other works exploring transformer architecture for point cloud learning [57], [78], [280] have also emerged recently.”
- Point Transformer V2: Grouped Vector Attention and Partition-based Pooling (2022, methodology): “During the research upsurge of vision transformers, at almost the same period, Zhao et al. [1] and Guo et al. [21] published their explorations of applying attention to point cloud understanding, becoming pioneers in this direction.”
- ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding (2022, neutral): “PointNet [35] 89.2 86.0 PointCNN [22] 92.2 - SpiderCNN [55] 92.4 - PointConv [50] 92.5 - Point Transformer [60] 92.8 - KPConv [44] 92.9 - DGCNN [47] 92.9 90.2 PCT [14] 93.2 - RS-CNN* [26] 93.6 - GDANet [54] 93.8 - GBNet [38] 93.8 91.0 MTVN [15] 93.8 92.0 RPNet [41] 94.1 - CurveNet [52] 94.”
- Visual attention network (2022, background): “Various deep self-attention networks ( a.k.a., vision transformers) [13], [15], [20], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64] have achieved significantly better performance than the mainstream CNNs on different visual tasks, showing the huge potential of…”
- PointCLIP: Point Cloud Understanding by CLIP (2021, background): “With 3D sensing technology developing rapidly, the growing demand for processing 3D point cloud data has boosted many advanced deep models with better local feature ag-gregator [30, 32, 49], geometry modeling [20, 39, 40] and projection-based processing [20, 34, 48].”
- Transformer-Based Visual Segmentation: A Survey (2023, result): “OneFormer [271] performs the best on ADE20 K.”
- Localization Distillation for Object Detection (2022, methodology): “Cross architecture distillation using recent state-of-the-art classification models [11], [20], [25], [43], [67] as teachers is also an interesting direction to explore.”
- A novel time–frequency Transformer based on self–attention mechanism and its application in fault diagnosis of rolling bearings (2021, background): “[46] put forward a point cloud Transformer (PCT) for point cloud learning to solve 3D computer vision problems.”