Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules

By Johannes Klicpera, S. Giri, Johannes T. Margraf, Stephan Günnemann. 2020

430 citations recorded by the source corpus.

Many important tasks in chemistry revolve around molecules during reactions. This requires predictions far from the equilibrium, while most recent work in machine learning for molecules has been focused on equilibrium or near-equilibrium states. In this paper we aim to extend this scope in three ways. First, we propose the DimeNet++ model, which is 8x faster and 10% more accurate than the original DimeNet on the QM9 benchmark of equilibrium molecules. Second, we validate DimeNet++ on highly reactive molecules by developing the challenging COLL dataset, which contains distorted configurations of small molecules during collisions. Finally, we investigate ensembling and mean-variance estimation for uncertainty quantification with the goal of accelerating the exploration of the vast space of non-equilibrium structures. Our DimeNet++ implementation as well as the COLL dataset are available online.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Spectroscopy, Computational Chemistry & Drug Discovery, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Spectroscopy

In Computational Chemistry & Drug Discovery

In AI for Physical Sciences

Citations in the corpus

E(n) Equivariant Graph Neural Networks (2021, neutral): “We compare to NMP (Gilmer et al., 2017), Schnet (Sch¨utt et al., 2017b), Cormorant (Anderson et al., 2019), L1Net (Miller et al., 2020), LieConv (Finzi et al., 2020), DimeNet++ (Klicpera et al., 2020a), TFN (Thomas et al., 2018) and SE(3)-Tr. (Fuchs et al., 2020).”
Geometric and Physical Quantities improve E(3) Equivariant Message Passing (2021, methodology): “Methods such as Dimenet++ (Klicpera et al., 2020), SphereNet (Liu et al., 2021), and GemNet (Klicpera et al., 2021) incorporate relative orientation in a second-order message passing scheme that considers angles between the central point and neighbours-of-neighbours.”
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs (2022, methodology): “Machine learned models can accelerate the prediction of quantum properties of atomistic systems like molecules by learning approximations of ab initio calculations [29, 88, 37, 25, 4, 10, 49, 77, 70, 54, 51].”
Efficient and Accurate Physics-aware Multiplex Graph Neural Networks for 3D Small Molecules and Macromolecule Complexes (2022, methodology): “□ A.2 Baseline Descriptions • DimeNet [15] is a message passing neural network using directional • DimeNet++ [14] is an improved version of DimeNet with better • D-MPNN [44] is a message passing neural network that incorporates • CMPNN [32] is built based on D-MPNN and has a communicative • SIGN [18”
Geometric deep learning on molecular representations (2021, methodology): “When applied to quantum chemistry tasks, GNNs often use E(3)-invariant 3D information by including radial and angular information into the edge features of the graph [39, 63, 64, 72, 73], thereby improving the prediction accuracy of quantum chemical forces and energies for equilibrium and nonequilib”
GemNet: Universal Directional Graph Neural Networks for Molecules (2021, result): “We compare with the results reported by several state-of-the-art models: sGDML [10], FCHL19 [12], SchNet [53], DimeNet [35], DimeNet [34], SphereNet [39], NequIP [4], PaiNN [54], ForceNet [30], and SpinConv [55].”
Equivariant message passing for the prediction of tensorial properties and molecular spectra (2021, background): “To ensure rotational invariance of a predicted property, the message function is often restricted to depend only on rotationally invariant inputs such as distances 25,28 or angles 15,34 .”
Towards universal neural network potential for material discovery applicable to arbitrary combination of 45 elements (2021, methodology): “1 [8] E(3) Invariant MLP X SchNet [10] E(3) Invariant GNN X DimeNet++ [11, 12] E(3) Invariant GNN X PhysNet [13] E(3) Invariant GNN X Cormorant [14] SE(3) Equivariant GNN X X SE(3)-Transformer [15] SE(3) Equivariant GNN X X NequIP [16] SE(3) Equivariant GNN X X SpookyNet [17] SE(3) Equivariant GNN X”
Crystal Diffusion Variational Autoencoder for Periodic Material Generation (2021, methodology): “3 GNN architecture We use DimeNet++ adapted for periodicity [26, 27] as the encoder, which is SE(3) invariant to the input structure.”
EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations (2023, positive): “…neural networks [5,7,11–18 , 31– 38] use equivariant irreps features built from vector spaces of irreducible representations ( irreps Invariant GNNs. Prior works [4,8,43–51] extract invariant information from 3D atomistic graphs and operate on the resulting graphs augmented with invariant features.”
Atomistic Line Graph Neural Network for improved materials property predictions (2021, background): “Recently, there has been growing interest in the explicit incorporation of bond angles and other many-body features 17,20 .”
MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design (2022, methodology): “This research line includes SchNet [47], OrbNet [48], DimeNet [49], [50], TFN [51], SE(3)-Transformer [51], EGNN [52], SpinConv [53], SphereNet [54], EVFN [55], UNiTE [56], SEGNN [57], PaiNN [58], and GemNet [59].”
MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields (2022, methodology): “The DTNN [42], SchNet [41], HIP-NN [35], PhysNet [48], or DimeNet [20, 29] approaches could only come close to but not improve upon the atomic descriptor-based methods in terms of computational efficiency”
Learning local equivariant representations for large-scale atomistic dynamics (2022, neutral): “Schnet [25] 14 19 14 14 DimeNet++ [54] 6.”
OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs (2021, methodology): “As modeling 3D molecular graphs is a promising direction in graph ML (Schütt et al., 2017; Klicpera et al., 2020; Sanchez-Gonzalez et al., 2020; Hu et al., 2021), we have updated PCQM4M to PCQM4Mv2 to include DFT-calculated 3D structures for training molecules.”