Canonical Capsules: Unsupervised Capsules in Canonical Pose

By Weiwei Sun, Andrea Tagliasacchi, Boyang Deng, Sara Sabour, Soroosh Yazdani, Geoffrey E. Hinton, Kwang Moo Yi. 2020

44 citations recorded by the source corpus.

We propose an unsupervised capsule architecture for 3D point clouds. We compute capsule decompositions of objects through permutation-equivariant attention, and self-supervise the process by training with pairs of randomly rotated objects. Our key idea is to aggregate the attention masks into semantic keypoints, and use these to supervise a decomposition that satisfies the capsule invariance/equivariance properties. This not only enables the training of a semantically consistent decomposition, but also allows us to learn a canonicalization operation that enables object-centric reasoning. In doing so, we require neither classification labels nor manually-aligned training datasets to train. Yet, by learning an object-centric representation in an unsupervised manner, our method outperforms the state-of-the-art on 3D point cloud reconstruction, registration, and unsupervised classification. We will release the code and dataset to reproduce our results as soon as the paper is published.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

How to Represent Part-Whole Hierarchies in a Neural Network (2021, methodology): “One way to avoid using a mixture for modeling the diﬀerent types of part is to have a set of identical, “universal” capsules, each of which contains enough knowledge to model any type of part [Locatello et al., 2020, Srivastava et al., 2019, Sun et al., 2020b].”
Unsupervised part representation by Flow Capsules (2020, background): “Several works learn capsule autoencoders for 3D objects from point clouds (Srivastava et al., 2019; Zhao et al., 2019; Sun et al., 2020).”
Vector Neurons: A General Framework for SO(3)-Equivariant Networks (2021, background): “However, while Sun et al. [31] learn category-level as well as multi-category pose estimation in a fully unsupervised fashion, the underlying equivariant backbone [30] is only equivariant by augmentation .”