Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds

By Yongming Rao, Jiwen Lu, Jie Zhou. 2020

150 citations recorded by the source corpus.

Local and global patterns of an object are closely related. Although each part of an object is incomplete, the underlying attributes about the object are shared among all parts, which makes reasoning the whole object from a single part possible. We hypothesize that a powerful representation of a 3D object should model the attributes that are shared between parts and the whole object, and distinguishable from other objects. Based on this hypothesis, we propose to learn point cloud representation by bidirectional reasoning between the local structures at different abstraction hierarchies and the global shape without human supervision. Experimental results on various benchmark datasets demonstrate the unsupervisedly learned representation is even better than supervised representation in discriminative power, generalization ability, and robustness. We show that unsupervisedly trained point cloud models can outperform their supervised counterparts on downstream classification tasks. Most notably, by simply increasing the channel width of an SSG PointNet++, our unsupervised model surpasses the state-of-the-art supervised methods on both synthetic and real-world 3D object classification datasets. We expect our observations to offer a new perspective on learning better representation from data structures instead of human annotations for point cloud understanding.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020, methodology): “Despite the dataset not playing to the strengths of our model (full SE(3)-invariance) and a much lower number of input points, the performance is competitive with models specifically designed for object classification - PointGLR [23], for instance, is pre-trained on the larger ModelNet40 dataset [41”
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following (2023, methodology): “Most self-supervised pre-training methods employ an encoder-decoder framework to encode point clouds into latent representations and then reconstruct the original data form [62, 66, 75].”
CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding (2022, methodology): “Recent line of works [17, 36, 42, 46, 48, 53, 64] introduce various pretext self-supervision tasks with the goal of learning rich semantic point attributes which eventually leads to high-level discriminative knowledge.”
Masked Autoencoders for Point Cloud Self-supervised Learning (2022, background): “SSL for Point Cloud SSL has also been widely studied for point cloud representation learning [41,58,48,36,1,20,52,34,13,51].”
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling (2021, background): “More recently, several studies suggested using SSL techniques for point cloud understanding [10,14,22,30,36,38,39,42,50,54,57].”