Sparse Unsupervised Capsules Generalize Better

By David Rawlinson, Abdelrahman Ahmed, Gideon Kowadlo. 2018

52 citations recorded by the source corpus.

We show that unsupervised training of latent capsule layers using only the reconstruction loss, without masking to select the correct output class, causes a loss of equivariances and other desirable capsule qualities. This implies that supervised capsules networks can't be very deep. Unsupervised sparsening of latent capsule layer activity both restores these qualities and appears to generalize better than supervised masking, while potentially enabling deeper capsules networks. We train a sparse, unsupervised capsules network of similar geometry to Sabour et al (2017) on MNIST, and then test classification accuracy on affNIST using an SVM layer. Accuracy is improved from benchmark 79% to 90%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Stacked Capsule Autoencoders (2019, methodology): “This task was used by Rawlinson et al., 2018 to evaluate Sparse Unsupervised Capsules, which achieved 90 .”
Building Deep, Equivariant Capsule Networks (2019, unknown): “Rawlinson et al. [2018], Rawlinson et. al showed that the supervised loss in capsule networks was important to their exhibiting equivariance.”
Unsupervised part representation by Flow Capsules (2020, background): “Unsupervised capsule learning has been explored by (Ko-siorek et al., 2019) and (Rawlinson et al., 2018).”
Radiologist-Level COVID-19 Detection Using CT Scans with Detail-Oriented Capsule Networks (2020, background): “However, Cap-sNets with dynamic or EM routing experience unstable training when the number of layers increases [25, 26].”