Geometric Capsule Autoencoders for 3D Point Clouds

By Nitish Srivastava, Hanlin Goh, Ruslan Salakhutdinov. 2019

22 citations recorded by the source corpus.

We propose a method to learn object representations from 3D point clouds using bundles of geometrically interpretable hidden units, which we call geometric capsules. Each geometric capsule represents a visual entity, such as an object or a part, and consists of two components: a pose and a feature. The pose encodes where the entity is, while the feature encodes what it is. We use these capsules to construct a Geometric Capsule Autoencoder that learns to group 3D points into parts (small local surfaces), and these parts into the whole object, in an unsupervised manner. Our novel Multi-View Agreement voting mechanism is used to discover an object's canonical pose and its pose-invariant feature vector. Using the ShapeNet and ModelNet40 datasets, we analyze the properties of the learned representations and show the benefits of having multiple votes agree. We perform alignment and retrieval of arbitrarily rotated objects -- tasks that evaluate our model's object identification and canonical pose recovery capabilities -- and obtained insightful results.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

How to Represent Part-Whole Hierarchies in a Neural Network (2021, methodology): “One way to avoid using a mixture for modeling the diﬀerent types of part is to have a set of identical, “universal” capsules, each of which contains enough knowledge to model any type of part [Locatello et al., 2020, Srivastava et al., 2019, Sun et al., 2020b].”
Unsupervised part representation by Flow Capsules (2020, background): “(Srivastava et al., 2019; Zhao et al., 2019) have proposed unsupervised learning of capsule autoencoders for 3D objects from point clouds.”
Canonical Capsules: Unsupervised Capsules in Canonical Pose (2020, methodology): “Note that this simpliﬁes the design (and training) of the encoder E , which only needs to be invariant, rather than equivariant [44, 41].”