Dynamic Routing Between Capsules

By Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton. 2017

5,162 citations recorded by the source corpus.

A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning (2021, positive): “In the rest of this section, we present experiments on the following real MTL datasets: MovieLens, Multi-MNIST, Multi-Fashion MNIST, and on a real-world, large-scale recommender system.”
Object-Centric Learning with Slot Attention (2020, methodology): “Iterative routing Our iterative attention mechanism shares similarlities with iterative routing mechanisms typically employed in variants of Capsule Networks [12, 13, 62].”
Local Relation Networks for Image Recognition (2019, methodology): “Channel sharing Following [24], the local relation layer uses channel sharing in aggregation computation, where multiple channels share the same aggregation weights.”
Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs (2021, neutral): “One example of how relative position is used in the design of computer vision algorithms is in capsule networks [28], which encode every object with a pose (i.e., translation and rotation).”
Dynamic Neural Networks: A Survey (2021, background): “, both the computational graph and the network parameters are fixed once trained, which may limit their representation power, efficiency and interpretability [11], [12], [13], [14].”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “Architectural alternatives such as capsule networks [70] can also organize the information in a network into pieces that disentangle and represent higher-level concepts.”
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “Let us take Capsule Networks (Sabour et al., 2017) and the Slot Attention module (Locatello et al., 2020) in the computer vision field as examples: these modules try to create object-centric and part-centric representations inside the latent space of the network.”
Weight Agnostic Neural Networks (2019, methodology): “As a proof of concept, we investigate how WANNs perform on the MNIST dataset [55], an image classiﬁcation task which has been a focus of human-led architecture design for decades [14, 57, 93].”
Unsupervised Learning of Neural Networks to Explain Neural Networks (2018, background): “For example, capsule nets [25] and interpretable RCNN [37] learned interpretable middle-layer features.”
Interpreting CNNs via Decision Trees (2018, methodology): “The capsule net [25] used a dynamic routing mechanism to parse the entire object into a parsing tree of capsules.”
Interpretable Convolutional Neural Networks (2017, methodology): “[20] proposed a capsule model, which used a dynamic routing mechanism to parse the entire object into a parsing tree of capsules, and each capsule may encode a specific meaning.”
Visual interpretability for deep learning: a survey (2018, background): “[Sabour et al., 2017] has designed novel neural units, namely capsules, in order to substitute for traditional neural units to construct a capsule network.”
Greedy AutoAugment (2019, background): “Another example of this approach is CapsuleNet, which tries to find the relevant pose information automatically [17], [18], [19].”
On the Binding Problem in Artificial Neural Networks (2020, background): “, 2015), energy-based models (Mordatch, 2019), and capsules (Sabour et al., 2017; Kosiorek et al., 2019).”
Recurrent Independent Mechanisms (2019, background): “, 2018) and the preceding Dynamic Capsules (Sabour et al., 2017) use the poses of parts and learned part→ object relationships to vote for the poses of objects.”
Counterfactuals uncover the modular structure of deep generative models (2018, methodology): “ected by ﬁlters to compute an explanatory graph [Zhang et al., 2017a]. Furthermore, explicitly enforcing modularity before training has been investigated recently with Capsule networks architectures [Sabour et al., 2017]. The study of generative processes differ signiﬁcantly, as the effect of change”
An overview of deep learning in medical imaging focusing on MRI (2018, neutral): “There are new ways of constructing more data-efficient deep neural networks on the horizon, for example by encoding more domain-specific elements in the neural network structure as in the capsule systems of [340,341], which adds viewpoint invariance.”
Effects of Degradations on Deep Neural Network Architectures (2018, methodology): “The final 16dimensional capsule layer is same as in [16], but to reduce the number of parameters, we remove the decoder module of basic capsule architecture and minimize only the marginal loss.”