Geometric and Physical Quantities improve E(3) Equivariant Message Passing

By Johannes Brandstetter, Rob D. Hesselink, Elise van der Pol, Erik J. Bekkers, Max Welling. 2021

323 citations recorded by the source corpus.

Including covariant information, such as position, force, velocity or spin is important in many tasks in computational physics and chemistry. We introduce Steerable E(3) Equivariant Graph Neural Networks (SEGNNs) that generalise equivariant graph networks, such that node and edge attributes are not restricted to invariant scalars, but can contain covariant information, such as vectors or tensors. This model, composed of steerable MLPs, is able to incorporate geometric and physical information in both the message and update functions. Through the definition of steerable node attributes, the MLPs provide a new class of activation functions for general use with steerable feature fields. We discuss ours and related work through the lens of equivariant non-linear convolutions, which further allows us to pin-point the successful components of SEGNNs: non-linear message aggregation improves upon classic linear (steerable) point convolutions; steerable messages improve upon recent equivariant graph networks that send invariant messages. We demonstrate the effectiveness of our method on several tasks in computational physics and chemistry and provide extensive ablation studies.

Subfields: Artificial Intelligence, Materials Chemistry

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

Citations in the corpus

State-specific protein–ligand complex structure prediction with a multiscale deep generative model (2022, positive): “2e and Supplementary Information Section E) is inspired by previous works on 3D graph and attentional neural networks for point clouds 61,62 , rigid-body simulations 63 and biopolymer representation learning 1,64–66 .”
EquiBind: Geometric Deep Learning for Drug Binding Structure Prediction (2022, methodology): “…Euclidean symmetries into GNNs. Injecting Euclidean 3D transformations into geometric DL models has become possible using equivariant message passing layers (Cohen & Welling, 2016; Thomas et al., 2018; Fuchs et al., 2020; Satorras et al., 2021; Brandstetter et al., 2021; Batzner et al., 2021).”
The design space of E(3)-equivariant atom-centred interatomic potentials (2022, unknown): “We denote the set of neighbors of an atom i , i.e., atoms with the cutoﬀ distance, by N ( i ).”
A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection (2023, background): “…by physical constraints, promising performance has been achieved thanks to the incorporation of inductive biases from known physical laws [190], [191] and architectures that maintain the symmetries of the underlying system, such as equivariance to rotations and translations [189], [192], [193].”
e3nn: Euclidean Neural Networks (2022, methodology): “So far, e3nn and Euclidean neural networks have been applied to diverse applications: finding symmetry breaking order parameters [28], predicting molecular properties [18], prediction of phonon density of states [6], prediction of the interatomic potentials [2], developing deep learning for diffusio”
EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations (2023, unknown): “Additionally, for the task of predicting DFT calculations of energies and forces on the large-scale OC20 dataset, invariant GNNs have been surpassed by equivariant GNNs recently.”
Message Passing Neural PDE Solvers (2022, methodology): “Parameters θ PDE are inserted into the message passing similar to Brandstetter et al. (2021) Decoder After message passing, we use a shallow 1D convolutional network with shared weights across spatial locations to output the K next timestep predictions at grid point x i .”
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs (2022, methodology): “SEGNN [5] utilizes non-linear message passing networks [29, 62] for irreps features.”
MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design (2022, methodology): “This research line includes SchNet [47], OrbNet [48], DimeNet [49], [50], TFN [51], SE(3)-Transformer [51], EGNN [52], Spin-Conv [53], SphereNet [54], EVFN [55], UNiTE [56], SEGNN [57], PaiNN [58], and GemNet [59].”
MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields (2022, methodology): “Models such as NequIP [5], equivariant transformer [45], PaiNN [43], or SEGNNs [8], in addition to invariant scalars, employ equivariant internal features that transform like vectors or tensors.”
Learning local equivariant representations for large-scale atomistic dynamics (2022, methodology): “Notably, the NequIP model [15], followed by several other equivariant implementations [26, 27, 33, 34, 40], demonstrated unprecedentedly low error on a large range of molecular and materials systems, accurately describes structural and kinetic properties of complex materials, and exhibits…”
Dynamic-Backbone Protein-Ligand Structure Prediction with Multiscale Generative Diffusion Models (2022, methodology): “The architecture of ESDM (Figure1e) is inspired by prior works on 3D graph and attentional neural networks for point clouds [37, 38], rigid-body simulations [39] and biopolymer representation learning [9, 40–42].”