Modeling Relational Data with Graph Convolutional Networks
By Michael Sejr Schlichtkrull, Thomas Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, Max Welling. 2017
6,128 citations recorded by the source corpus.
Knowledge graphs enable a wide variety of applications, including question answering and information retrieval. Despite the great effort invested in their creation and maintenance, even the largest (e.g., Yago, DBPedia or Wikidata) remain incomplete. We introduce Relational Graph Convolutional Networks (R-GCNs) and apply them to two standard knowledge base completion tasks: Link prediction (recovery of missing facts, i.e. subject-predicate-object triples) and entity classification (recovery of missing entity attributes). R-GCNs are related to a recent class of neural networks operating on graphs, and are developed specifically to handle the highly multi-relational data characteristic of realistic knowledge bases. We demonstrate the effectiveness of R-GCNs as a stand-alone model for entity classification. We further show that factorization models for link prediction such as DistMult can be significantly improved through the use of an R-GCN encoder model to accumulate evidence over multiple inference steps in the graph, demonstrating a large improvement of 29.8% on FB15k-237 over a decoder-only baseline.
Subfields: Artificial Intelligence, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Percy Liang
- Yoshua Bengio
- Jianfeng Gao
- Luke S. Zettlemoyer
- Quoc V. Le
- Ruslan Salakhutdinov
- Yejin Choi
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Aleksandar Bojchevski
- Matthias Fey
- Marinka Zitnik
- Jian Tang
- Rex Ying
Citations in the corpus
- Stochastic Training of Graph Convolutional Networks (2017, background): “…& Welling, 2017), inductive node embedding (Hamilton et al., 2017a), link prediction (Kipf & Welling, 2016; Berg et al., 2017) and knowledge graphs (Schlichtkrull et al., 2017), outperforming multi-layer perceptron (MLP) models that do not use the graph structure and graph embedding approaches…”
- Why Should We Add Early Exits to Neural Networks? (2020, background): “While a number of innovative components have been proposed recently (such as attention layers [66], neural ODEs [16], and graph modules [58]), the vast majority of deep networks is designed as a sequential stack of (differentiable) layers, trained by”
- Fast Graph Representation Learning with PyTorch Geometric (2019, methodology): “For learning on point clouds, manifolds and graphs with multi-dimensional edge features, we provide the relational GCN operator from Schlichtkrull et al. (2018), PointNet++ (Qi et al., 2017), PointCNN (Li et al., 2018), and the continuous kernel-based methods MPNN (Gilmer et al., 2017), MoNet…”
- On the Binding Problem in Artificial Neural Networks (2020, background): “…applied to a wide variety of graph-structured data including social networks (Hamilton et al., 2017), citation networks (Kipf and Welling, 2017), 3D surfaces (Litany et al., 2018), knowledge base completion tasks (Schlichtkrull et al., 2018), and bio-chemical modeling (Atwood and Towsley, 2016).”
- Flexible Neural Representation for Physics Prediction (2018, positive): “Our work draws inspiration from and extends that of Chang et al. [11] and Battaglia et al. [4], which in turn use ideas from graph-based neural networks [39, 44, 9, 30, 22, 14, 13, 24, 8, 40].”
- Hierarchical Graph Representation Learning with Differentiable Pooling (2018, methodology): “GNNs have been applied to a wide variety of tasks, including node classification [16, 21], link prediction [31], graph classification [7, 11, 39], and chemoinformatics [14, 19, 27, 28, 32].”
- Predict then Propagate: Graph Neural Networks meet Personalized PageRank (2018, background): “…Hamilton et al., 2017; Veliˇckovi´c et al., 2018), random walks (Abu-El-Haija et al., 2018a; Ying et al., 2018; Li et al., 2018), edge features (Kearnes et al., 2016; Gilmer et al., 2017; Schlichtkrull et al., 2018) and making it more scalable on large graphs (Chen et al., 2018; Ying et al., 2018).”
- Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022, methodology): “Based on these results, we can find: As for non-pre-training methods, text-based methods ( i.e., TextCNN and TextRCNN) outperform GNN-based methods ( i.e., GAT and R-GCN).”
- Recurrent Event Network for Reasoning over Temporal Knowledge Graphs (2019, methodology): “Thus, we adopt the block-diagonal decomposition [20], where each relation-specific weight matrix is decomposed into a block-diagonal and low-dimensional matrix. and B is the number of basis matrices.”
- Attention-based Graph Neural Network for Semi-supervised Learning (2018, background): “We give a typical example of a GNN in Section 3, but several diverse variations have been proposed in [10, 17, 29, 24, 39, 8, 15, 13, 3, 32, 38, 23, 37, 38].”
- Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020, background): “…entities from text, e.g., extracting the relation “ the president of ” from the given sentence “ Newton served as the president of the Royal Society ”, which could serve as external resource for various downstream applications (Dong et al., 2015; Xiong et al., 2017; Schlichtkrull et al., 2018).”
- Convolutional 2D Knowledge Graph Embeddings (2017, methodology): “Relational GCNs (R-GCNs) [26] are a generalisation of GCNs developed for dealing with highly multi-relational data such as knowledge graphs – we include them in our experimental evaluations.”
- Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks (2019, methodology): “Based on the different techniques for aggregating neighborhood information, there are graph convolutional networks [19], GraphSAGE [20], gated graph recurrent networks [18] and their variants.”