Gated Graph Sequence Neural Networks
By Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard S. Zemel. 2015
3,648 citations recorded by the source corpus.
Abstract: Graph-structured data appears frequently in domains including chemistry, natural language semantics, social networks, and knowledge bases. In this work, we study feature learning techniques for graph-structured inputs. Our starting point is previous work on Graph Neural Networks (Scarselli et al., 2009), which we modify to use gated recurrent units and modern optimization techniques and then extend to output sequences. The result is a flexible and broadly useful class of neural network models that has favorable inductive biases relative to purely sequence-based models (e.g., LSTMs) when the problem is graph-structured. We demonstrate the capabilities on some simple AI (bAbI) and graph algorithm learning tasks. We then show it achieves state-of-the-art performance on a problem from program verification, in which subgraphs need to be matched to abstract data structures.
Subfields: Artificial Intelligence, Materials Chemistry, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Percy Liang
- Jianfeng Gao
- Sergey Levine
- Quoc V. Le
- Ruslan Salakhutdinov
- Luke S. Zettlemoyer
In Materials Chemistry
- William L. Hamilton
- Jure Leskovec
- Tommi S. Jaakkola
- Regina Barzilay
- Rex Ying
- Wengong Jin
- Stephan Günnemann
- Max Welling
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Matthias Fey
- Michael M. Bronstein
- Rex Ying
- Aleksandar Bojchevski
- Marinka Zitnik
Citations in the corpus
- Semi-Supervised Classification with Graph Convolutional Networks (2016, background): “ network. Their framework requires the repeated application of contraction maps as propagation functions until node representations reach a stable fixed point. This restriction was later alleviated in Li et al. (2016) by introducing modern practices for recurrent neural network training to the origin”
- Graph Attention Networks (2017, methodology): “This idea was adopted and improved by Li et al. (2016), which propose to use gated recurrent units (Cho et al., 2014) in the propagation step.”
- Inductive Representation Learning on Large Graphs (2017, neutral): “There are also a number of recent neural network approaches to supervised learning over graph structures [7, 10, 21, 31].”
- Object-Centric Learning with Slot Attention (2020, background): “[55] Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel.”
- Learning Convolutional Neural Networks for Graphs (2016, background): “Gated Graph Sequence Neural Networks modify GNNs to use gated recurrent units and to output sequences (Li et al., 2015).”
- VAIN: Attentional Multi-agent Predictive Modeling (2017, neutral): “The seminal works in graph neural networks were presented by Scarselli et al. [12, 13] and Li et al. [14].”
- A simple neural network module for relational reasoning (2017, neutral): “Although several other models supporting relation-centric computation have been proposed, such as Graph Neural Neworks, Gated Graph Sequence Neural Netoworks, and Interaction Networks, [20, 13, 2], RNs are simpler, more exclusively focused on general relation reasoning, and easier to integrate withi”
- Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs (2016, background): “This work remained practically unnoticed and has been rediscovered only recently [24, 37].”
- Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “From this perspective, the model in (Li et al., 2015) can be considered as an extension of (Scarselli et al.”
- Neural Logic Machines (2019, methodology): “As a comparison with graph neural networks, Li et al. (2016) achieved 99% accuracy on the bAbI task 19.”
- Interaction Networks for Learning about Objects, Relations and Physics (2016, background): “4 , string MSE 1 .”
- DeepCoder: Learning to Write Programs (2016, methodology): “Li et al. (2016) use graph neural networks (GNNs) to predict logical descriptions from program states, focusing on data structure shapes instead of numerical and list data.”
- Neural Message Passing for Quantum Chemistry (2017, methodology): “Our implementation of MPNNs in general operates on directed graphs with a separate message channel for incoming and outgoing edges, in which case the incoming message mv is the concatenation of minv and m out v , this was also used in Li et al. (2016).”
- Fast Graph Representation Learning with PyTorch Geometric (2019, methodology): “We additionaly offer more sophisticated methods such as set-to-set (Vinyals et al., 2016), sort pooling (Zhang et al., 2018) or the global soft attention layer from Li et al. (2016).”