Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network

By Risi Kondor, Zhen Lin, Shubhendu Trivedi. 2018

315 citations recorded by the source corpus.

Recent work by Cohen \emph{et al.} has achieved state-of-the-art results for learning spherical images in a rotation invariant way by using ideas from group representation theory and noncommutative harmonic analysis. In this paper we propose a generalization of this work that generally exhibits improved performace, but from an implementation point of view is actually simpler. An unusual feature of the proposed architecture is that it uses the Clebsch--Gordan transform as its only source of nonlinearity, thus avoiding repeated forward and backward Fourier transforms. The underlying ideas of the paper generalize to constructing neural networks that are invariant to the action of other compact groups.

Subfields: Applied Mathematics, Computer Vision and Pattern Recognition, Geophysics, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Computer Vision and Pattern Recognition

In Geophysics

In Model Architectures

Citations in the corpus

Directional Message Passing for Molecular Graphs (2020, methodology): “, 2018) or by using a fully Fourier space model (Kondor et al., 2018; Anderson et al., 2019).”
General E(2)-Equivariant Steerable CNNs (2019, methodology): “This result is particularly important for SE(3) ∼= (R,+) o SO(3)-equivariant CNNs whose feature fields are often transforming under the irreps of SO(3) [39, 2, 33, 20, 34].”
Cormorant: Covariant Molecular Neural Networks (2019, methodology): “, 2018] and [Kondor, 2018] were to first to use the Clebsch–Gordan transform inside rotationally covariant neural networks for learning physical systems, while [Kondor et al., 2018] showed that in spherical CNNs the Clebsch–Gordan transform is sufficient to serve as the sole source of nonlinearity.”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, methodology): “…networks have been proposed for permutation-equivariant analysis and prediction of sets (Zaheer et al., 2017; Hartford et al., 2018), graphs (Kondor et al., 2018b; Hy et al., 2018; Maron et al., 2019), translations and rotations of the plane and 3D space (Oyallon & Mallat, 2015; Cohen &…”
On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups (2018, methodology): “In particular, we argue for Fourier space representations, similar to those that have appeared in (Worrall et al., 2017; Cohen et al., 2018; Kondor et al., 2018)), and, even more recently, since the submission of the original version of the present paper in (Thomas et al.”
DeepCMB: Lensing Reconstruction of the Cosmic Microwave Background with Deep Neural Networks (2018, background): “…the Universe, as shown through a number of experiments over the past two decades — e.g., COBE, Boomerang, WMAP, Planck, SPT, ACT (Mather et al., 1994; Lange et al., 2001; Bennett et al., 2013; Planck Collaboration et al., 2018; ∗Corresponding author Email address: jcaldeira@uchicago.edu (J.…”
GemNet: Universal Directional Graph Neural Networks for Molecules (2021, background): “These directions define an equivariant mesh that circumvents the aliasing effects that would arise from fixed grids [36].”
e3nn: Euclidean Neural Networks (2022, unknown): “[30, 33, 17].”
Re-evaluating evaluation (2018, background): “The question admits many answers depending on the context, see e.g. [73–79].”
Geometric and Physical Quantities improve E(3) Equivariant Message Passing (2021, methodology): “with arbitrary multiplicities ml for each type, we implement what is called generalised steerable convolutions, of which the SO(3) convolutions are a special case with ml = (2l + 1) (Kondor et al., 2018).”
E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials (2021, methodology): “A series of methods has been introduced to realize rotationally equivariant neural networks [13, 20, 28– 30].”
DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications (2018, background): “…log N pix , plus N pix matrix multiplications of size N pix for a total cost of O N 3 / 2 pix operations. leveraged to avoid SHTs between layers (it does a non-linear transformation in the spectral domain), the transform itself costs O ( N 3 / 2 pix ), for no reduction of overall complexity [55].”
Learning local equivariant representations for large-scale atomistic dynamics (2022, background): “More recently, however, a class of models known as equivariant neural networks [36–39] have been developed which can act directly on non-invariant geometric inputs, such as displacement vectors, in a symmetry-respecting way.”
A Group-Theoretic Framework for Data Augmentation (2019, background): “Starting from [30, 45, 61, 14], a line of research focuses on designing (or learning) neural net architectures that are invariant to certain symmetries [31, 27, 70, 20, 22, 19], equivariant to SO(3) group [21, 28, 29], and others [58, 42, 69, 41].”
Vector Neurons: A General Framework for SO(3)-Equivariant Networks (2021, background): “A whole body of work is built on the theory of SO(3) representations [32, 16, 11, 36, 1] – most of these works rely on the concept of convolution with steerable kernel bases.”