On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups

By Risi Kondor, Shubhendu Trivedi. 2018

590 citations recorded by the source corpus.

Convolutional neural networks have been extremely successful in the image recognition domain because they ensure equivariance to translations. There have been many recent attempts to generalize this framework to other domains, including graphs and data lying on manifolds. In this paper we give a rigorous, theoretical treatment of convolution and equivariance in neural networks with respect to not just translations, but the action of any compact group. Our main result is to prove that (given some natural constraints) convolutional structure is not just a sufficient, but also a necessary condition for equivariance to the action of a compact group. Our exposition makes use of concepts from representation theory and noncommutative harmonic analysis and derives new generalized convolution formulae.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Radiology, Nuclear Medicine and Imaging, Model Architectures, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Radiology, Nuclear Medicine and Imaging

In Model Architectures

In Statistical Learning Theory

Citations in the corpus

Relational inductive biases, deep learning, and graph networks (2018, background): “Kondor et al. (2018) suggested that covariance 7 (Cohen and Welling, 2016; Kondor and Trivedi, 2018), rather than invariance to permutations of the nodes and edges is preferable, and proposed “covariant compositional networks” which can preserve structural information, and allow it to be ignored…”
Group Equivariant Stand-Alone Self-Attention For Vision (2020, methodology): “Interestingly, this observation is inline with the main statement of Kondor & Trivedi (2018) for (group) convolutions: “ the group convolution on (cid:71) is the only (unique) (cid:71) -equivariant linear map ”.”
Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data (2020, background): “G , · For any u ∈ the convolution of g and f on G at u is given by and Trivedi, 2018; Cohen et al., 2019)”
Set2Graph: Learning Graphs From Sets (2020, background): “Set-to-graph functions are well-defined if they satisfy a property called equivariance, and therefore the set-to-graph problem is an instance of the bigger class of equivariant learning (Cohen and Welling, 2016; Ravanbakhsh et al., 2017; Kondor and Trivedi, 2018).”
General E(2)-Equivariant Steerable CNNs (2019, background): “As a result, we are able to implement a wide range of equivariant models, covering regular GCNNs [6, 7, 8, 9, 10, 11], classical Steerable CNNs [1], Harmonic Networks [12], gated Harmonic Networks [2], Vector Field Networks [13], Scattering Transforms [14, 15, 16, 17, 18] and entirely new architectu”
Cormorant: Covariant Molecular Neural Networks (2019, methodology): “Kondor and Trivedi [2018] discuss the generalization of convolution to the action of compact groups (such as, in our case, rotations) and prove that the only possible linear operation that is covariant with the group action, is what, in terms of SO(3) –vectors, corresponds to multiplying each F…”
Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds (2018, background): “Further discussion of related theory can be found in [20], [21], and [22].”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, methodology): “These models be classiﬁed as Regular G-CNNs (which use scalar and regular features) (Kondor & Trivedi, 2018) and Steerable G-CNNs (general ﬁelds) (Cohen et al., 2018c;a).”
Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network (2018, background): “In particular, it has been shown that whenever there is a compact group G acting on the inputs of a neural network, there is a natural notion of Fourier transformation with respect to G, yielding a sequence of Fourier matrices {F s ` }` at each layer, and the linear operation at layer s will be equi”
Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, methodology): “Our method uses 3D reconstruction to project visual data onto a spatiallyconsistent representation as input, with which it is able to better exploit equivariance [13, 14] for inductive biases that are present within the geometric symmetries [15] of the data for more efficient learning.”
N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials (2018, methodology): “One particular architecture, which makes the part-based aspect of neural modeling very explicit, is that of compositional networks (comp-nets), introduced in (Kondor et al., 2018).”
A General Theory of Equivariant CNNs on Homogeneous Spaces (2018, background): “This map is analogous to the lifting defined by [12] for scalar fields (i.”
CubeNet: Equivariance to 3D Rotation and Translation (2018, background): “Recently there has been an explosion of interest into CNNs with predefined transformation equivariances, beyond translation [8, 50, 11, 29, 16, 42, 36, 14, 19, 31, 18, 22, 15, 55, 49, 26, 25, 48, 33, 28, 9].”
3D G-CNNs for Pulmonary Nodule Detection (2018, background): “For the general theory, we refer the reader to (15; 16; 17; 18).”
Deep Scale-spaces: Equivariance Over Scale (2019, unknown): “We call this a lift [Kondor and Trivedi, 2018], denoting lifted functions as f ↑ .”
Probabilistic Symmetries and Invariant Neural Networks (2020, methodology): “…of feed-forward networks (i.e., linear maps composed with pointwise nonlinearities) that are invariant under the action of discrete groups (Shawe-Taylor, 1989; Ravanbakhsh et al., 2017), finite linear groups (Wood and Shawe-Taylor, 1996), or compact groups (Kondor and Trivedi, 2018).”
A Group-Theoretic Framework for Data Augmentation (2019, background): “Starting from [30, 45, 61, 14], a line of research focuses on designing (or learning) neural net architectures that are invariant to certain symmetries [31, 27, 70, 20, 22, 19], equivariant to SO(3) group [21, 28, 29], and others [58, 42, 69, 41].”