3D Point Capsule Networks

By Yongheng Zhao, Tolga Birdal, Haowen Deng, Federico Tombari. Computer Vision and Pattern Recognition, 2018

365 citations recorded by the source corpus.

In this paper, we propose 3D point-capsule networks, an auto-encoder designed to process sparse 3D point clouds while preserving spatial arrangements of the input data. 3D capsule networks arise as a direct consequence of our unified formulation of the common 3D auto-encoders. The dynamic routing scheme and the peculiar 2D latent space deployed by our capsule networks bring in improvements for several common point cloud-related tasks, such as object classification, object reconstruction and part segmentation as substantiated by our extensive evaluations. Moreover, it enables new applications such as part interpolation and replacement.

View Paper: https://www.semanticscholar.org/paper/29c05c8a2f24ddf9572019b396801ad9bb21e884

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Learning for 3D Point Clouds: A Survey (2019, background): “[112] proposed a permutation-invariant PS(2)-Net to incorporate local structures and global context from point clouds.”
Canonical Capsules: Unsupervised Capsules in Canonical Pose (2020, methodology): “Of particular relevance to our work, are methods that apply capsule networks to 3D input data [64, 65, 46], but note these methods are not unsupervised, as they either rely on classification supervision [65], or on datasets that present a significant inductive bias in the form of pre-alignment [64].”
Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions (2020, background): “An active research area within 3D scene understanding focuses on 3D semantic segmentation [34, 4, 36, 7, 38, 12] and object detection and classification [41, 34, 35, 52].”
Geometric Capsule Autoencoders for 3D Point Clouds (2019, methodology): “The 3D Point Capsule Network [22] showed the possibility of extracting semantically meaningful parts from 3D point clouds using capsule-based representations.”
Review: deep learning on 3D point clouds (2020, methodology): “3DPointCapsNet [69] proposed an approach that does not consider the local correlation between points, but region correlation is achieved using the novel dynamic routing procedure proposed by Sabour et al.”
Unsupervised part representation by Flow Capsules (2020, background): “(Srivastava et al., 2019; Zhao et al., 2019) have proposed unsupervised learning of capsule autoencoders for 3D objects from point clouds.”
CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding (2022, background): “Initial line of works exploit generative modelling using generative adversarial networks [1, 15, 57] and auto-encoders [11, 16, 26, 65, 71], which aims to reconstruct a given input point cloud with varying architectural designs.”
PF-Net: Point Fractal Network for 3D Point Cloud Completion (2020, methodology): “3D point capsule networks [24, 25] surpasses the performance of other methods and becomes the state-of-the-art Auto-Encoder processing point cloud, especially in the domain of point cloud reconstruction.”
Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds (2020, methodology): “Self-Reconstruction: Self-reconstruction, or point autoencoding, is a widely used technique for unsupervised point cloud representation learning [1,8,14,26,47,55,56].”
Unsupervised Point Cloud Pre-training via Occlusion Completion (2020, methodology): “We compare OcCo with Jig-saw [42], and cTree [44] since it outperforms previous un-supervised methods [1, 54, 62, 59] as well as supervised variants [38, 30, 37, 53].”
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends (2023, unknown): “Furthermore, AEshavebeenusedforpointcloudsinworkslike[113],[114], [152],[153],whilecapsulenetworkshavebeenappliedtopoint clouddatain[154].”
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, background): “rast boosts performance. Relative improvement increases with scarcer training data and on less frequent classes. methods IoU (1% data) IoU (5% data) IoU (100% data) SO-Net[36] 64.0 69.0 - PointCapsNet[86] 67.0 70.0 - Multitask Unsupervised[22] 68.2 77.7 - Train from scratch 71.8 79.3 84.7 PointConst”
Fully Convolutional Geometric Features (2019, unknown): “…3.210 Folding [33] 0.613 0.087 0.023 0.010 512 0.352 PPFNet [7] 0.623 0.108 0.003 0.005 64 2.257 PPF-Fold [6] 0.718 0.105 0.731 0.104 512 0.794 DirectReg [8] 0.746 0.094 - - 512 0.794 CapsuleNet [37] 0.807 0.062 0.807 0.062 512 1.208 PerfectMatch [11] tance thresholds, and inlier recall thresholds.”