Stacked Capsule Autoencoders

By Adam R. Kosiorek, Sara Sabour, Yee Whye Teh, Geoffrey E. Hinton. 2019

280 citations recorded by the source corpus.

Objects are composed of a set of geometrically organized parts. We introduce an unsupervised capsule autoencoder (SCAE), which explicitly uses geometric relationships between parts to reason about objects. Since these relationships do not depend on the viewpoint, our model is robust to viewpoint changes. SCAE consists of two stages. In the first stage, the model predicts presences and poses of part templates directly from the image and tries to reconstruct the image by appropriately arranging the templates. In the second stage, SCAE predicts parameters of a few object capsules, which are then used to reconstruct part poses. Inference in this model is amortized and performed by off-the-shelf neural encoders, unlike in previous capsule networks. We find that object capsule presences are highly informative of the object class, which leads to state-of-the-art results for unsupervised classification on SVHN (55%) and MNIST (98.7%). The code is available at this https URL

Subfields: Computer Vision and Pattern Recognition, Generative Models, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Model Architectures

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “For example, SCAE can only discover stroke-like structures that are uninterpretable to humans on the Street View House Numbers (SVHN) dataset (Netzer et al., 2011) (see Figure 11).”
On the Binding Problem in Artificial Neural Networks (2020, background): “, 2015), energy-based models (Mordatch, 2019), and capsules (Sabour et al., 2017; Kosiorek et al., 2019).”
How to Represent Part-Whole Hierarchies in a Neural Network (2021, methodology): “With considerable eﬀort, models that use capsules have achieved some suc-7 cesses in supervised and unsupervised learning on small datasets [Sabour et al., 2017, Hinton et al., 2018, Kosiorek et al., 2019], but they have not scaled well to larger datasets [Barham and Isard, 2019].”
GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations (2019, background): “Stacked Capsule Autoencoders (Kosiorek et al. [17]) discover geometric relationships between objects and their parts by using an afﬁne-aware decoder.”
Capsules with Inverted Dot-Product Attention Routing (2020, methodology): “This hierarchical relationship modeling has spurred a lot of interest in designing capsules and their routing algorithms (Sabour et al., 2017; Hinton et al., 2018; Wang & Liu, 2018; Zhang et al., 2018; Li et al., 2018; Rajasegaran et al., 2019; Kosiorek et al., 2019).”
Logic Tensor Networks (2020, result): “Secondly, LTN should be compared in large-scale practical use cases with other recent efforts to add structure to neural networks such as the Neuro-Symbolic Concept Learner [43] and high-level capsules which were used recently to learn the part-of relation [38], similarly to LTN in [18].”
Subclass Distillation (2020, methodology): “An alternative method for unsupervised clustering with deep neural networks that is not based on the contrastive loss can be found in Kosiorek et al. (2019), where they use capsule networks to directly learn MNIST and CIFAR-10 classes.”
Unsupervised part representation by Flow Capsules (2020, methodology): “Unsupervised capsule learning with an image reconstruction loss for part discovery has been explored by (Kosiorek et al., 2019) and (Rawlinson et al., 2018).”
Canonical Capsules: Unsupervised Capsules in Canonical Pose (2020, methodology): “In this paper, we take inspiration from the recent Stacked Capsule Auto-Encoders [29], where the authors have shown how capsule-style reasoning is effective as far as primary capsules can be trained in an unsupervised fashion – by only using unsupervised reconstruction losses.”
Generative Adversarial Transformers (2021, background): “Works such as Capsule networks [62] leverage ideas from psychology about Gestalt principles [34, 64], perceptual grouping [6] or analysis-by-synthesis [4], and like us, introduce ways to piece together visual elements to discover compound entities or, in the cases of Set Transformers [50] and A(2)-N”
Deep learning for AI (2021, background): “15,50,59 First, we learn a layer of feature detectors whose activities allow us to re-construct the input.”
Neurosymbolic AI: the 3rd wave (2020, unknown): “The recently-proposed stacked capsule autoencoders (Kosiorek et al. 2019) and neurosymbolic approaches such as Logic Tensor Networks (Serafini and d’Avila Garcez 2016) as well as other semi-supervised or weakly-supervised approaches revive the important stance of modularity in neural computation.”
Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence (2021, background): “A recent study, for example, shows how two different design approaches to ML algorithms (convolutional neural networks [44] and capsule networks [45, 46]) differently explain why an image is classified one way rather than another, and in some cases, choosing a different mathematical design for an al”
Radiologist-Level COVID-19 Detection Using CT Scans with Detail-Oriented Capsule Networks (2020, background): “This property has prompted several research groups to develop new capsule designs and routing algorithms [24, 30, 25, 31].”
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2020, unknown): “[199], called stacked capsule autoencoder (SCAE).”
Biological underpinnings for lifelong learning machines (2022, background): “On a more abstract level, few-shot complex object learning can be framed as the combination of parts into wholes based on their relationships, which are captured in capsule networks 186 , and has led to the formation of the Omniglot dataset 187 .”
Geometric Capsule Autoencoders for 3D Point Clouds (2019, methodology): “[11] follows this approach in their encoder but during decoder a separate decoder network per object index is used, making the feature representation place-coded.”