Vector Neurons: A General Framework for SO(3)-Equivariant Networks

By Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas J. Guibas. IEEE International Conference on Computer Vision, 2021

435 citations recorded by the source corpus.

Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds. Yet most proposed methods either use complex mathematical tools that may limit their accessibility, or are tied to specific input data types and network architectures. In this paper, we introduce a general framework built on top of what we call Vector Neuron representations for creating SO (3) -equivariant neural networks for pointcloud processing. Extending neurons from 1D scalars to 3D vectors, our vector neurons enable a simple mapping of SO (3) actions to latent spaces thereby providing a framework for building equivariance in common neural operations – including linear layers, non-linearities, pooling, and normalizations. Due to their simplicity, vector neurons are versatile and, as we demonstrate, can be incorporated into diverse network architecture backbones, allowing them to process geometry inputs in arbitrary poses. Despite its simplicity, our method performs comparably well in accuracy and generalization with other more complex and specialized state-of-the-art methods on classification and segmentation tasks. We also show for the first time a rotation equivariant reconstruction network. Source code is available at https://github.com/FlyingGiraffe/vnn.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design

View Paper: https://www.semanticscholar.org/paper/7d24930875b50602f79dc0464b60e50c4336998f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

Citations in the corpus

Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation (2021, methodology): “Furthermore, their functional nature enables the principled incorporation of symmetries, such as SO(3) equivariance [7, 50].”
Canonical Capsules: Unsupervised Capsules in Canonical Pose (2020, background): “Convolutional Neural Networks lack equivariance to rigid transformations, despite their pivotal role in describing the structure of the 3D scene behind a 2D image.”
Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets (2022, positive): “To address this problem, Deng et al. (2021) proposes vector neurons which extends 1D scalar neurons to 3D vectors and also defines a set of equivariant operations in the vector space.”
Geometric and Physical Quantities improve E(3) Equivariant Message Passing (2021, background): “A different but also fully message passing based approach can be found in Geometric Vector Perceptrons (GVP) (Jing et al., 2020), Vector Neurons (Deng et al., 2021), and PaiNN (Sch¨utt et al., 2021).”