Neural Message Passing for Quantum Chemistry
By Justin Gilmer, Sam Schoenholz, Patrick F. Riley, Oriol Vinyals, George E. Dahl. International Conference on Machine Learning, 2017
9,235 citations recorded by the source corpus.
Supervised learning on molecules has incredible potential to be useful in chemistry, drug discovery, and materials science. Luckily, several promising and closely related neural network models invariant to molecular symmetries have already been described in the literature. These models learn a message passing algorithm and aggregation procedure to compute a function of their entire input graph. At this point, the next step is to find a particularly effective variant of this general approach and apply it to chemical prediction benchmarks until we either solve them or reach the limits of the approach. In this paper, we reformulate existing models into a single common framework we call Message Passing Neural Networks (MPNNs) and explore additional novel variations within this framework. Using MPNNs we demonstrate state of the art results on an important molecular property prediction benchmark; these results are strong enough that we believe future work should focus on datasets with larger molecules or more accurate ground truth labels.
Subfields: Computational Theory and Mathematics, Materials Chemistry, Physical and Theoretical Chemistry, Graph Learning
View Paper: https://arxiv.org/abs/1704.01212
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Theory and Mathematics
- Tommi S. Jaakkola
- Regina Barzilay
- Sergey Levine
- Jure Leskovec
- William L. Hamilton
- Gisbert Schneider
- Jian Tang
- Max Welling
In Materials Chemistry
- Tommi S. Jaakkola
- Regina Barzilay
- Tess E. Smidt
- William L. Hamilton
- Jure Leskovec
- Max Welling
- Jian Tang
- Shitong Luo
In Physical and Theoretical Chemistry
- Tommi S. Jaakkola
- Jian Tang
- Regina Barzilay
- Tess E. Smidt
- Jure Leskovec
- Sam Schoenholz
- Shitong Luo
- Risi Kondor
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Matthias Fey
- Rex Ying
- Jian Tang
- Michael M. Bronstein
- Marinka Zitnik
Citations in the corpus
- Neural Logic Machines (2019, background): “As a representative, Gilmer et al. (2017) proposes a message passing modeling for unifying various graph neural networks and graph convolution networks.”
- Strategies for Pre-training Graph Neural Networks (2019, methodology): “Modern GNNs use a neighborhood aggregation approach, where representation of node v is iteratively updated by aggregating representations of v ’s neighboring nodes and edges (Gilmer et al., 2017).”
- E(n) Equivariant Graph Neural Networks (2021, methodology): “(Gilmer et al., 2017) presented a message passing setting (or Graph Neural Network) for quantum chemistry, this method is permutation equivariant but not translation or rotation equivariant.”
- Directional Message Passing for Molecular Graphs (2020, methodology): “With the advent of graph neural networks (GNNs) this approach has recently experienced a small revolution, since they do not require any form of manual feature engineering and significantly outperform previous models (Gilmer et al., 2017; Schütt et al., 2017).”
- Robust deep learning based protein sequence design using ProteinMPNN (2022, methodology): “We used encoder-decoder message passing neural networks for this task (1, 20, 25), see Figure 1.”
- Fast Graph Representation Learning with PyTorch Geometric (2019, methodology): “Similar to the concepts of convolutional and pooling layers on regular domains, GNNs are able to (hierarchically) extract localized embeddings by passing, transforming, and aggregating information (Bronstein et al., 2017; Gilmer et al., 2017; Battaglia et al., 2018; Ying et al., 2018).”
- Compositional Fairness Constraints for Graph Embeddings (2019, background): “A wide variety of techniques have been proposed to learn low-dimensional embeddings of nodes and edges in graphs [10], including methods based on matrix decomposition [22], random-walk based objectives [18], and graph neural networks [8].”
- Auto-GNN: Neural architecture search of graph neural networks (2019, methodology): “Following the popular message passing strategy (Gilmer et al., 2017), GNNs are stacked by a series of graph convoutional layers.”
- On the Binding Problem in Artificial Neural Networks (2020, background): “Message Passing Neural Networks Message Passing Neural Networks (MPNNs; Gilmer et al., 2017) iteratively update the node representations of a given graph by exchanging messages along its edges (until convergence)20.”
- Learning to Simulate Complex Physics with Graph Networks (2020, background): “A GN maps an input graph to an output graph with the same structure but potentially different node, edge, and graph-level attributes, and can be trained to learn a form of message-passing (Gilmer et al., 2017), where latent information is propagated between nodes via the edges.”
- Recurrent Independent Mechanisms (2019, background): “…scenes, and motion-capture data, and multiagent systems (Scarselli et al., 2008; Bronstein et al., 2017; Watters et al., 2017; Raposo et al., 2017; Santoro et al., 2017; Gilmer et al., 2017; Van Steenkiste et al., 2018; Kipf et al., 2018; Battaglia et al., 2018; Tacchetti et al., 2018).”
- Structured agents for physical construction (2019, background): “, 2014), implemented with graph neural networks (Scarselli et al., 2009; Bronstein et al., 2017; Gilmer et al., 2017; Battaglia et al., 2018).”
- Learning to predict the cosmological structure formation (2018, background): “This new paradigm is also significantly impacting a variety of domains in the sciences, from biology (27, 28) to chemistry (29, 30) and physics (31, 32).”
- Relational inductive biases, deep learning, and graph networks (2018, methodology): “Recently, a class of models has arisen at the intersection of deep learning and structured approaches, which focuses on approaches for reasoning about explicitly structured data, in particular graphs (e.g. Scarselli et al., 2009b; Bronstein et al., 2017; Gilmer et al., 2017; Wang et al., 2018c; Li e”
- Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions (2018, background): “The proposed interaction function Υ R-NEM can be seen as a type of Message Passing Neural Network (MPNN; Gilmer et al. (2017)) that incorporates a variant of neighborhood attention (Duan et al., 2017).”
- Differentiable Physics-informed Graph Networks (2019, background): “One branch of these studies is a spectral approach to aggregate locally connected attributes (Bruna et al., 2014; Defferrard et al., 2016; Kipf & Welling, 2017) and another branch is a spatial approach (or message passing method) (Gilmer et al., 2017; Monti et al., 2017; Hamilton et al., 2017).”
- Position-aware Graph Neural Networks (2019, background): “…according to trainable attention weights (Velickovic et al., 2018); Message Passing Neural Networks further incorporate edge information when doing the aggregation (Gilmer et al., 2017); And, Graph Networks further consider global graph information during aggregation (Battaglia et al., 2018).”