Steerable CNNs

By Taco Cohen, Max Welling. 2016

591 citations recorded by the source corpus.

It has long been recognized that the invariance and equivariance properties of a representation are critically important for success in many vision tasks. In this paper we present Steerable Convolutional Neural Networks, an efficient and flexible class of equivariant convolutional networks. We show that steerable CNNs achieve state of the art results on the CIFAR image classification benchmark. The mathematical theory of steerable representations reveals a type system in which any steerable representation is a composition of elementary feature types, each one associated with a particular kind of symmetry. We show how the parameter cost of a steerable filter bank depends on the types of the input and output features, and show how to use this knowledge to construct CNNs that utilize parameters effectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

E(n) Equivariant Graph Neural Networks (2021, background): “Group equivariant neural networks have demonstrated their effectiveness in a wide variety of tasks (Cohen & Welling, 2016; 2017; Weiler & Cesa, 2019; Rezende et al., 2019; Romero & Cordonnier, 2021).”
Directional Message Passing for Molecular Graphs (2020, methodology): “We can do so by generating a separate embedding m ji for each atom i and neighbor j by applying the same learned ﬁlter in the direction of each neighboring atom (in contrast to equivariant CNNs, which apply ﬁlters in ﬁxed, global directions).”
Stacked Capsule Autoencoders (2019, background): “Spatial Transformers ( ST ; Jaderberg et al., 2015) apply afﬁne transformations to the image sampling grid while steerable networks (Cohen and Welling, 2017; Jacobsen et al., 2017) dynamically change convolutional ﬁlters.”
Covariant Compositional Networks For Learning Graphs (2018, background): “…an active ﬁeld of research, they differ from classical CNNs in a fundamental way: the internal feature representations in CNNs are equivariant to such transformations of the inputs as translation and rotations (Cohen & Welling, 2016a;b), the internal representations in MPNNs are fully invariant.”
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020, background): “Furthermore, if (Tg, Sg) are linear and the map φ is also linear, then a very rich and developed theory already exists for finding φ [6].”
Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data (2020, methodology): “One approach to constructing equivariant CNNs, ﬁrst introduced in Cohen and Welling (2016a), is to use standard convolutional kernels and transform them or the feature maps for each of the elements in the group.”
Set2Graph: Learning Graphs From Sets (2020, background): “…Veliˇckovi´c et al., 2017; Xu et al., 2019; Kondor et al., 2018; Maron et al., 2019b;a), learning images with rotational and reﬂectional symmetries (Cohen & Welling, 2016a;b; Dieleman et al., 2016; Worrall et al., 2017), learning functions on spheres (Cohen et al., 2018; Esteves et al., 2017)…”
General E(2)-Equivariant Steerable CNNs (2019, methodology): “As a result, we are able to implement a wide range of equivariant models, covering regular GCNNs [6, 7, 8, 9, 10, 11], classical Steerable CNNs [1], Harmonic Networks [12], gated Harmonic Networks [2], Vector Field Networks [13], Scattering Transforms [14, 15, 16, 17, 18] and entirely new…”
Cormorant: Covariant Molecular Neural Networks (2019, background): “At the same time, there have been signiﬁcant advances in our understanding of the equivariance and covariance properties of neural networks, starting with [Cohen and Welling, 2016a,b] in the context of traditional convolutional neural nets (CNNs).”
Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds (2018, background): “An introduction to the concepts of steerability and equivariance in the context of neural networks can be found in Cohen and Welling [19], which focuses on discrete symmetries.”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, positive): “As shown experimentally in (Cohen & Welling, 2016; 2017) and in this paper, it is often more effective to use convolutions that preserve orientation information.”
Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network (2018, background): “Recently, a number of papers have appeared that examine equivariance from the theoretical point of view, motivated by the understanding that the natural way to generalize convolutional networks to other types of data will likely lead through generalizing the notion of equivariance itself to other tr”
RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks (2018, methodology): “In particular, steer-able ﬁlters were used in [5, 36, 37].”
Spherical CNNs (2018, background): “Another interesting generalization is the development of a Steerable CNN for the sphere (Cohen and Welling, 2017), which would make it possible to analyze vector ﬁelds such as global wind directions, as well as other sections of vector bundles over the sphere.”
Harmonic Networks: Deep Translation and Rotation Equivariance (2016, background): “More recently they generalized this theory to all group-structured transformations in [4], but they only demonstrated applications on finite groups—an extension to continuous transformations would require a treatment on anti-aliasing and bandlimiting.”
Understanding Image Representations by Measuring Their Equivariance and Equivalence (2014, background): “In fact, some more recent works have attempted to design steerable CNN representations (Cohen and Welling 2017) for CIFAR10 dataset (Krizhevsky and Hinton 2009).”
On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups (2018, background): “This property is called equivariance (Cohen & Welling, 2016).”
N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials (2018, background): “…et al., 2017; Kondor et al., 2018), manifolds (Monti et al., 2016; Masci et al., 2015), and other domains (Bruna & Mallat, 2013; Cohen et al., 2018), as well as the question of generalizing the concept of equivariance (covariance) in general (Cohen & Welling, 2016; 2017; Kondor & Trivedi, 2018).”
Learning Steerable Filters for Rotation Equivariant CNNs (2017, methodology): “Group-convolutional networks were proposed by Co-hen and Welling [2] who considered four ﬁlter orientations.”