Gauge Equivariant Convolutional Networks and the Icosahedral CNN

By Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, Max Welling. 2019

474 citations recorded by the source corpus.

The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging problems that exhibit symmetries. Here we show how this principle can be extended beyond global symmetries to local gauge transformations. This enables the development of a very general class of convolutional neural networks on manifolds that depend only on the intrinsic geometry, and which includes many popular methods from equivariant and geometric deep learning. We implement gauge equivariant CNNs for signals defined on the surface of the icosahedron, which provides a reasonable approximation of the sphere. By choosing to work with this very regular manifold, we are able to implement the gauge equivariant convolution using a single conv2d call, making it a highly scalable and practical alternative to Spherical CNNs. Using this method, we demonstrate substantial improvements over previous methods on the task of segmenting omnidirectional images and global climate patterns.

Subfields: Computational Mechanics, Computational Theory and Mathematics, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computational Theory and Mathematics

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Directional Message Passing for Molecular Graphs (2020, background): “Following work has generalized this principle to spheres (Cohen et al., 2018), molecules (Thomas et al., 2018), volumetric data (Weiler et al., 2018), and general manifolds (Cohen et al., 2019).”
Group Equivariant Stand-Alone Self-Attention For Vision (2020, background): “…et al., 2018; Lenssen et al., 2018; Graham et al., 2020) to spherical rotations (Cohen et al., 2018; Worrall & Brostow, 2018; Weiler et al., 2018a; Cohen et al., 2019b), scaling (Marcos et al., 2018; Worrall & Welling, 2019; Sosnovik et al., 2020; Romero et al., 2020b) and more general symmetry…”
Isometric Transformation Invariant and Equivariant Graph Convolutional Networks (2020, background): “ohen & Welling (2016) for discrete groups. Subsequent research works generalized it to continuous groups (Cohen et al., 2018), three-dimensional data (Weiler et al., 2018), and general manifolds (Cohen et al., 2019). Concerning point clouds, Thomas et al. (2018); Kondor (2018) discussed the way ”
General E(2)-Equivariant Steerable CNNs (2019, methodology): “Our results can therefore readily be transferred to e.g. spherical CNNs [19, 5, 20, 21, 22, 23] or more general models of geometric deep learning [24, 25, 26, 27].”
Cormorant: Covariant Molecular Neural Networks (2019, background): “Kondor and Trivedi [2018] gave a complete characterization of what operations are allowable in Fourier space neural networks to preserve covariance, and Cohen et al generalized the framework even further to arbitrary gauge ﬁelds [Cohen et al., 2019].”
A General Theory of Equivariant CNNs on Homogeneous Spaces (2018, methodology): “We note that the associated bundle construction works for any principal H-bundle, nog just p : G→ G/H , which suggests a direction for further generalization [11].”
On the "steerability" of generative adversarial networks (2019, background): “In general, this goal also captures the idea in equivariance, in which transformations in the input space result in equivalent transformations in the output space (c.f. Hinton et al. (2011); Cohen et al. (2019); Lenc & Vedaldi (2015)).”
DeepSphere: a graph-based spherical CNN (2020, background): “Compared to Cohen et al. (2019)’s conclusion, it is surprising that S2R does worse than DeepSphere (which is limited to S2S).”
Attentive Group Equivariant Convolutional Networks (2020, methodology): “2): Resultantly, the computational cost of a group convolution is roughly equivalent to that of a spatial convolution with a ﬁlter bank of size N ˜ c × | H | (Cohen & Welling, 2016a; Worrall & Welling, 2019; Cohen et al., 2019b).”
B-Spline CNNs on Lie Groups (2019, methodology): “In other words, given the definition of G cross-correlation in (6), denoted with ?, and gauge correlation in (18) or (Cohen et al., 2019, Eq.”
A Survey of Sound Source Localization with Deep Learning Methods (2021, unknown): “In this survey paper, we do not address the problem of tracking on its own, which is usually done in a separate algorithm using the sequence of DoA estimates obtained by applying SSL on successive time windows [67].”
Wavelet Networks: Scale Equivariant Learning From Raw Waveforms (2020, methodology): “[15] Taco S Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling.”
Multipole Graph Neural Operator for Parametric Partial Differential Equations (2020, methodology): “Because of GNNs’ flexibility, they can be used to model convolution with different discretization and geometry [13, 14].”
Universal Equivariant Multilayer Perceptrons (2020, background): “[6] Taco S Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling.”
On Learning Sets of Symmetric Elements (2020, background): “From a theoretical point of view, several papers studied the properties of equivariant layers (Ravanbakhsh et al., 2017; Kondor & Trivedi, 2018; Cohen et al., 2019a) and characterized the expressive power of models that use such layers (Yarotsky, 2018; Maron et al., 2019c; Keriven & Peyr´e, 2019;…”