Learning Steerable Filters for Rotation Equivariant CNNs

By Maurice Weiler, Fred A. Hamprecht, Martin Storath. 2017

448 citations recorded by the source corpus.

In many machine learning tasks it is desirable that a model's prediction transforms in an equivariant way under transformations of its input. Convolutional neural networks (CNNs) implement translational equivariance by construction; for other transformations, however, they are compelled to learn the proper mapping. In this work, we develop Steerable Filter CNNs (SFCNNs) which achieve joint equivariance under translations and rotations by design. The proposed architecture employs steerable filters to efficiently compute orientation dependent responses for many orientations without suffering interpolation artifacts from filter rotation. We utilize group convolutions which guarantee an equivariant mapping. In addition, we generalize He's weight initialization scheme to filters which are defined as a linear combination of a system of atomic filters. Numerical experiments show a substantial enhancement of the sample complexity with a growing number of sampled filter orientations and confirm that the network generalizes learned patterns over orientations. The proposed approach achieves state-of-the-art on the rotated MNIST benchmark and on the ISBI 2012 2D EM segmentation challenge.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Geophysics, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Geophysics

In Model Architectures

Citations in the corpus

Rotation Equivariant CNNs for Digital Pathology (2018, methodology): “A standard convolution 2 (denoted ∗ ) of an input f with ﬁlter ψ is deﬁned as: G-CNNs are a generalization of CNNs that are equivariant under more general symmetry groups, such as the group G = p 4 of 90 ◦ rotations, or G = p 4 m which additionally includes reﬂection.”
Group Equivariant Stand-Alone Self-Attention For Vision (2020, background): “Similar behaviour is seen for steerable G-CNNs, e.g., Weiler et al. (2018b).”
General E(2)-Equivariant Steerable CNNs (2019, methodology): “Similar approaches have been followed in simpler settings for the cases of CN in [7], SO(2) in [12] and SO(3) in [2].”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, background): “…Maron et al., 2019), translations and rotations of the plane and 3D space (Oyallon & Mallat, 2015; Cohen & Welling, 2016; 2017; Marcos et al., 2017; Weiler et al., 2018b;a; Worrall et al., 2017; Worrall & Brostow, 2018; Winkels & Cohen, 2018; Winkens et al., 2018; Thomas et al., 2018; Bekkers et…”
RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks (2018, methodology): “In particular, steer-able ﬁlters were used in [5, 36, 37].”
Spherical CNNs (2018, background): “With the exception of SO(2)-steerable networks (Worrall et al., 2017; Weiler et al., 2017), these networks are all limited to discrete groups, such as discrete rotations acting on planar images or permutations acting on point clouds.”
Alias-Free Generative Adversarial Networks (2021, background): “Group-equivariant CNNs aim to generalize the efﬁciency beneﬁts of translational weight sharing to, e.g., rotation [16, 65, 63, 62] and scale [64].”
A General Theory of Equivariant CNNs on Homogeneous Spaces (2018, background): “The sampling procedure requires particular attention since it might introduce aliasing artifacts [4, 6].”
CubeNet: Equivariance to 3D Rotation and Translation (2018, methodology): “Recently there has been an explosion of interest into CNNs with predefined transformation equivariances, beyond translation [8, 50, 11, 29, 16, 42, 36, 14, 19, 31, 18, 22, 15, 55, 49, 26, 25, 48, 33, 28, 9].”
B-Spline CNNs on Lie Groups (2019, methodology): “This is in line with findings in (Weiler et al., 2018b) where a fixed circular harmonic basis is used.”
Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data (2019, background): “…feature maps, several approaches have emerged to extend equivariance to, e.g., planar rotations (Dieleman et al., 2016; Marcos et al., 2017; Weiler et al., 2018; Li et al., 2018), spherical rotations (Cohen et al., 2018; Worrall & Brostow, 2018; Cohen et al., 2019), scaling (Marcos et al.,…”
Group Equivariant Capsule Networks (2018, background): “One way is to share weights over differently rotated ﬁlters or augment the input heavily by transformations [Yanzhao et al., 2017, Weiler et al., 2018].”
Deep Scale-spaces: Equivariance Over Scale (2019, unknown): “…al., 2017] and 3D [Weiler et al., 2018a, Kondor et al., 2018, Thomas et al., 2018] and discrete roto-translations in 2D [Cohen and Welling, 2016a, Weiler et al., 2018b, Bekkers et al., 2018, Hoogeboom et al., 2018] and 3D [Worrall et al., 2017], continuous rotations on the sphere [Esteves et…”
Scale equivariance in CNNs with vector fields (2018, background): “Equivariance to other transformations, chiefly rotation, through weight sharing has gathered a lot of attention in recent literature (Cohen & Welling, 2016; Dieleman et al., 2016; Ravanbakhsh et al., 2017; Weiler et al., 2017; Zhou et al., 2017).”
Equivariant message passing for the prediction of tensorial properties and molecular spectra (2021, background): “These components again make use gated equivariant nonlinearities 17 and show similarities to geometric vector perceptrons (GVP) 32 .”
Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness (2019, methodology): “the mathematical model for rotations and translations has led to carefully hand-engineered architectures that incorporate spatial invariance directly [19, 24, 8, 27, 44, 42, 12, 40].”
On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location (2020, methodology): “…versions of the training data there are methods speciﬁcally designed to learn geometric transformations in an equivariant or invariant representation [7, 53, 60] where examples include rotation [19, 69, 102, 103, 110], scale [68, 92, 99, 104, 106] and other transformations [17, 26, 38, 57, 90].”