GemNet: Universal Directional Graph Neural Networks for Molecules

By Johannes Klicpera, Florian Becker, Stephan Günnemann. Neural Information Processing Systems, 2021

633 citations recorded by the source corpus.

Effectively predicting molecular interactions has the potential to accelerate molecular dynamics by multiple orders of magnitude and thus revolutionize chemical simulations. Graph neural networks (GNNs) have recently shown great successes for this task, overtaking classical methods based on fixed molecular kernels. However, they still appear very limited from a theoretical perspective, since regular GNNs cannot distinguish certain types of graphs. In this work we close this gap between theory and practice. We show that GNNs with spherical representations are indeed universal approximators for predictions that are invariant to translation, and equivariant to permutation and rotation. We then discretize such GNNs via directed edge embeddings and two-hop message passing, and incorporate multiple structural improvements to arrive at the geometric message passing neural network (GemNet). We demonstrate the benefits of the proposed changes in multiple ablation studies. GemNet outperforms previous models on the COLL, MD17, and OC20 datasets by 34%, 41%, and 20%, respectively, and performs especially well on the most challenging molecules. Our implementation is available online.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Spectroscopy

View Paper: https://www.semanticscholar.org/paper/5542d0ff99767f75f8c8a329fc3d88d73ff470c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Spectroscopy

Citations in the corpus

The design space of E(3)-equivariant atom-centred interatomic potentials (2022, neutral): “We denote the set of neighbors of an atom i , i.e., atoms with the cutoﬀ distance, by N ( i ).”
Crystal Diffusion Variational Autoencoder for Periodic Material Generation (2021, methodology): “Our encoder and decoder PGNNs directly use GNN architectures developed for the OCP (Klicpera et al., 2020b; 2021; Shuaibi et al., 2021; Godwin et al., 2021), which are also closely related to SE(3) equivariant networks (Thomas et al., 2018; Fuchs et al., 2020).”
EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations (2023, positive): “Comparing to the closest available EquifomerV2 point, GemNet-dT [50] is 1 .”
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs (2022, methodology): “The former leverages geometric tensors like vectors for equivariant node features [72, 80, 43, 23, 4, 5, 51], and the latter augments graphs with invariant information such as distances and angles extracted from 3D graphs [64, 26, 25, 48, 68, 42].”
E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials (2021, methodology): “Finally, we compare to a series of other methods including ACE [42], SpookyNet [25], and GemNet [24] as well as other equivariant neural networks such as PaiNN [22], NewtonNet [23], and UNiTE [26].”
MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design (2022, methodology): “This research line includes SchNet [47], OrbNet [48], DimeNet [49], [50], TFN [51], SE(3)-Transformer [51], EGNN [52], Spin-Conv [53], SphereNet [54], EVFN [55], UNiTE [56], SEGNN [57], PaiNN [58], and GemNet [59].”
MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields (2022, methodology): “MACE Allegro[36] BOTNet[3] NequIP[5] GemNet(T/Q)[30] ACE[33] FCHL[18] GAP[2] ANI[19] PaiNN[43] DimeNet[29] NewtonNet[24] ACE[33] the state of the art for some molecules, particularly those with the highest errors.”
Protein Representation Learning by Geometric Structure Pretraining (2022, methodology): “Besides, angles and dihedrals between adjacent edges are also important features that reﬂect the relative position between residues Klicpera et al. (2021).”
Uni-Mol: A Universal 3D Molecular Representation Learning Framework (2023, methodology): “A series of SE(3) models are proposed, such as SchNet [30], tensor field networks [73], SE(3) Transformer [74], DimmNet [31], equivariant graph neural networks (EGNN) [39], GemNet [32] and SphereNet [33].”
Learning local equivariant representations for large-scale atomistic dynamics (2022, neutral): “Molecule FCHL19 [13, 46] UNiTE [26] GAP [6] ANI-pretrained [47, 48] ANI-random [47, 48] ACE [12] GemNet-(T/Q) [49] NequIP (l=3) [15] Allegro”
Geometric and Physical Quantities improve E(3) Equivariant Message Passing (2021, methodology): “Methods such as Dimenet++ (Klicpera et al., 2020), SphereNet (Liu et al., 2021), and GemNet (Klicpera et al., 2021) incorporate relative orientation in a second-order message passing scheme that considers angles between the central point and neighbours-of-neighbours.”