On Learning Sets of Symmetric Elements

By Haggai Maron, Or Litany, Gal Chechik, Ethan Fetaya. 2020

152 citations recorded by the source corpus.

Learning from unordered sets is a fundamental learning setup, recently attracting increasing attention. Research in this area has focused on the case where elements of the set are represented by feature vectors, and far less emphasis has been given to the common case where set elements themselves adhere to their own symmetries. That case is relevant to numerous applications, from deblurring image bursts to multi-view 3D shape recognition and reconstruction. In this paper, we present a principled approach to learning sets of general symmetric elements. We first characterize the space of linear layers that are equivariant both to element reordering and to the inherent symmetries of elements, like translation in the case of images. We further show that networks that are composed of these layers, called Deep Sets for Symmetric Elements layers (DSS), are universal approximators of both invariant and equivariant functions. DSS layers are also straightforward to implement. Finally, we show that they improve over existing set-learning architectures in a series of experiments with images, graphs, and point-clouds.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Networks from the Principle of Rate Reduction (2020, background): “The recent work of Zaheer et al. (2017); Maron et al. (2020) characterize properties of networks and operators for set permutation groups.”
Vector Neurons: A General Framework for SO(3)-Equivariant Networks (2021, methodology): “Inspired by Maron et al. [22], we produce a matrix T n for each element by concatenating its feature V n ∈ R C × 3 with the global mean V := 1 N n V n ∈ R C × 3 and running through a vector neuron MLP with a target number of channel C (cid:48) = 3 : Finally we deﬁne our invariant layer by:”
GemNet: Universal Directional Graph Neural Networks for Molecules (2021, background): “Maron et al. [44] proved universality for models invariant to S n and equivariant to an additional symmetry.”
Universal Equivariant Multilayer Perceptrons (2020, background): “Universality results for some of these models exists (Zaheer et al., 2017; Segol & Lipman, 2019; Keriven & Peyré, 2019; Maron et al., 2020).”
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, background): “First, is the challenge of processing unordered sets [47, 50, 80, 39].”