Directional Message Passing for Molecular Graphs

By Johannes Klicpera, Janek Groß, Stephan Günnemann. International Conference on Learning Representations, 2020

1,129 citations recorded by the source corpus.

Graph neural networks have recently achieved great successes in predicting quantum mechanical properties of molecules. These models represent a molecule as a graph using only the distance between atoms (nodes) and not the spatial direction from one atom to another. However, directional information plays a central role in empirical potentials for molecules, e.g. in angular potentials. To alleviate this limitation we propose directional message passing, in which we embed the messages passed between atoms instead of the atoms themselves. Each message is associated with a direction in coordinate space. These directional message embeddings are rotationally equivariant since the associated directions rotate with the molecule. We propose a message passing scheme analogous to belief propagation, which uses the directional information by transforming messages based on the angle between them. Additionally, we use spherical Bessel functions to construct a theoretically well-founded, orthogonal radial basis that achieves better performance than the currently prevalent Gaussian radial basis functions while using more than 4x fewer parameters. We leverage these innovations to construct the directional message passing neural network (DimeNet). DimeNet outperforms previous GNNs on average by 77% on MD17 and by 41% on QM9.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Molecular Biology, Graph Learning, Computational Chemistry & Drug Discovery

View Paper: https://arxiv.org/abs/2003.03123

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Molecular Biology

In Graph Learning

In Computational Chemistry & Drug Discovery

Citations in the corpus

E(n) Equivariant Graph Neural Networks (2021, background): “Klicpera et al. (2020b;a) in addition to relative distances it includes a modiﬁed message passing scheme analogous to Belief Propagation that considers angles and directional information equivariant to rotations.”
Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules (2020, methodology): “To this end, we start from the recently proposed Directional Message Passing Neural Network (DimeNet) [7], which fulfills all of these properties except one: It is comparatively slow to compute.”
Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties (2020, background): “Cosine φC , Gaussian φG, and Bessel φB are defined in Equation 2, SchNet [4], or DimeNet [22] respectively.”
Isometric Transformation Invariant and Equivariant Graph Convolutional Networks (2020, methodology): “(15) As these approaches utilize the inner products of vectors in Rd, these operations correspond to the extractions of a relative distance and an angle of each pair of vertices, which are employed explicitly in Klicpera et al. (2020).”
Learning from Protein Structure with Geometric Vector Perceptrons (2020, methodology): “We also train and evaluate DimeNet, a recent 3D-aware GNN architecture which achieves state-of-the-art on many small-molecule tasks (Klicpera et al., 2019), on CASP 11-12.”
Learning Gradient Fields for Molecular Conformation Generation (2021, background): “Such design is a common practice in existing literature (Schütt et al., 2017; Hu et al., 2020; Klicpera et al., 2020), which is favored as it ensures several physical invariances, e.”
Machine Learning for High-entropy Alloys: Progress, Challenges and Opportunities (2022, methodology): “Different GCNN models have been proposed, such as SchNet [240], MEGNet [241], DimeNet [242], and GemNet [242], and their implementation can be found in the Pytorch geometric package [243].”
The design space of E(3)-equivariant atom-centred interatomic potentials (2022, neutral): “We denote the set of neighbors of an atom i , i.e., atoms with the cutoﬀ distance, by N ( i ).”
Generalization and Representational Limits of Graph Neural Networks (2020, background): “, DimeNet) perform significantly better (Klicpera et al., 2020).”
Weisfeiler and Leman go sparse: Towards scalable higher-order graph embeddings (2019, methodology): “We note here that our intent is not the beat state-of-the-art, physical knowledge-incorporating architectures, e.g., DimeNet (Klicpera et al., 2020) or Cormorant (Anderson et al., 2019), but to solely show the beneﬁts of the (local) higher-order architectures compared to the corresponding ( 1 -dimen”
Structure-aware Interactive Graph Neural Networks for the Prediction of Protein-Ligand Binding Affinity (2021, methodology): “• DimeNet [17] is a recent state-of-the-art model for small molecular graph learning using directional message passing scheme.”
EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations (2023, positive): “DimeNet series [4,46] use triplet representations of atoms to incorporate bond angles.”
Equivariant Diffusion for Molecule Generation in 3D (2022, background): “Behind the success of Alphafold in protein folding prediction (Jumper et al., 2021), an increasing body of literature develops deep learning models to analyze or synthesize (in silico) molecules (Simonovsky & Komodakis, 2018; Gebauer et al., 2019; Klicpera et al., 2020; Simm et al., 2021).”
Crystal Diffusion Variational Autoencoder for Periodic Material Generation (2021, methodology): “Our encoder and decoder PGNNs directly use GNN architectures developed for the OCP (Klicpera et al., 2020b; 2021; Shuaibi et al., 2021; Godwin et al., 2021), which are also closely related to SE(3) equivariant networks (Thomas et al., 2018; Fuchs et al., 2020).”