General E(2)-Equivariant Steerable CNNs

By Maurice Weiler, Gabriele Cesa. 2019

691 citations recorded by the source corpus.

The big empirical success of group equivariant networks has led in recent years to the sprouting of a great variety of equivariant network architectures. A particular focus has thereby been on rotation and reflection equivariant CNNs for planar images. Here we give a general description of E(2)-equivariant convolutions in the framework of Steerable CNNs. The theory of Steerable CNNs thereby yields constraints on the convolution kernels which depend on group representations describing the transformation laws of feature spaces. We show that these constraints for arbitrary group representations can be reduced to constraints under irreducible representations. A general solution of the kernel space constraint is given for arbitrary representations of the Euclidean group E(2) and its subgroups. We implement a wide range of previously proposed and entirely new equivariant network architectures and extensively compare their performances. E(2)-steerable convolutions are further shown to yield remarkable gains on CIFAR-10, CIFAR-100 and STL-10 when used as drop in replacement for non-equivariant convolutions.

Subfields: Computer Vision and Pattern Recognition, Neurology, Ocean Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Neurology

In Ocean Engineering

In Model Architectures

Citations in the corpus

E(n) Equivariant Graph Neural Networks (2021, background): “Group equivariant neural networks have demonstrated their effectiveness in a wide variety of tasks (Cohen & Welling, 2016; 2017; Weiler & Cesa, 2019; Rezende et al., 2019; Romero & Cordonnier, 2021).”
Group Equivariant Stand-Alone Self-Attention For Vision (2020, methodology): “It has been shown both theoretically (Ravanbakhsh, 2020) and empirically (Weiler & Cesa, 2019) that the most expressive class of (cid:71) -equivariant functions is given by functions that follow the regular representation of (cid:71) .”
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020, background): “This constraint has been pointed out in the equivariance literature to limit performance severely [36].”
Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data (2020, methodology): “Another approach, drawing on harmonic analysis, ﬁnds a basis of equivariant functions and parametrizes convolutional kernels in that basis (Worrall et al., 2017; Weiler and Cesa, 2019).”
A General Theory of Equivariant CNNs on Homogeneous Spaces (2018, background): “A comprehensive overview and comparison over equivariant nonlinearities can be found in [7].”
Attentive Group Equivariant Convolutional Networks (2020, positive): “…Brostow, 2018; Cohen et al., 2019b; Thomas et al., 2018; Kondor & Trivedi, 2018), scaling (Marcos et al., 2018; Worrall & Welling, 2019; Sos-novik et al., 2020) and general symmetry groups (Cohen & Welling, 2016a; Weiler & Cesa, 2019; Cohen et al., 2019a; Bekkers, 2020; Venkataraman et al., 2020).”
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2020, background): “Besides, the Euclidean group E(2) [32] and its subgroups as kernel space’s general solutions are given to adapt to the rotation and reflection of planar images.”
Universal Equivariant Multilayer Perceptrons (2020, background): “Some applications include equivariance to isometries of the Euclidean space [39, 42], and”
Wavelet Networks: Scale Equivariant Learning From Raw Waveforms (2020, methodology): “The usage of continuous smooth bases has proven advantageous for group convolutions [12, 76, 4, 75, 74, 13, 62, 3, 23].”
Geometric and Physical Quantities improve E(3) Equivariant Message Passing (2021, background): “…that designing CNNs to exploit additional symmetries via group convolutions has even further increased their performance (Cohen & Welling, 2016; 2017; Worrall et al., 2017; Cohen et al., 2018; Kondor & Trivedi, 2018; Weiler et al., 2018; Bekkers et al., 2018; Bekkers, 2019; Weiler & Cesa, 2019).”
Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images (2020, methodology): “In line with Weiler and Cesa [36], we use frequencies up to k = 0, 2, 3, 2 for j = 0, 1, 2, 3 in all 7 × 7 steerable basis filters.”
Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking (2021, background): “GNNs are validated in a variety of tasks such as particle system dynamics or conformation-based energy estimation (Weiler and Cesa, 2019; Rezende et al., 2019).”
ReDet: A Rotation-equivariant Detector for Aerial Object Detection (2021, methodology): “Specifically, we re-implement all layers of the backbone with rotation-equivariant networks based on e2cnn [32], including convolution, pooling, normalization, non linearities, etc.”