Universal Equivariant Multilayer Perceptrons

By Siamak Ravanbakhsh. 2020

61 citations recorded by the source corpus.

Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant MLPs with a single hidden layer. In particular, it is shown that having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance). A corollary is unconditional universality of equivariant MLPs for Abelian groups, such as CNNs with a single hidden layer. A second corollary is the universality of equivariant MLPs with a high-order hidden layer, where we give both group-agnostic bounds and means for calculating group-specific bounds on the order of hidden layer that guarantees universal equivariance (invariance).

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Group Equivariant Stand-Alone Self-Attention For Vision (2020, methodology): “Statement ( ii ) stems from the ﬁnding of Ravanbakhsh (2020) stating that functions induced by regular group representations are equivariant universal approximators provided global receptive ﬁelds.”
Probabilistic Symmetries and Invariant Neural Networks (2020, background): “…distributions, and in that sense they are related to the recent literature on universal approximations (as in Cybenko, 1989; Hornik, 1991) of invariant and equivariant functions (e.g., Yarotsky, 2018; Maron et al., 2019; Keriven and Peyré, 2019; Segol and Lipman, 2020; Ravanbakhsh, 2020).”