Inductive Representation Learning on Large Graphs
By William L. Hamilton, Zhitao Ying, Jure Leskovec. Neural Information Processing Systems, 2017
20,392 citations recorded by the source corpus.
Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.
Subfields: Artificial Intelligence, Molecular Biology, Representation & Self-Supervised Learning, Graph Learning
View Paper: https://arxiv.org/abs/1706.02216
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Percy Liang
- Jianfeng Gao
- Quoc V. Le
- Sergey Levine
- Ruslan Salakhutdinov
- Dacheng Tao
In Molecular Biology
- Jure Leskovec
- William L. Hamilton
- Tommi S. Jaakkola
- Stephan Günnemann
- Regina Barzilay
- Marinka Zitnik
- Rex Ying
- Matthias Fey
In Representation & Self-Supervised Learning
- Yoshua Bengio
- William L. Hamilton
- Jure Leskovec
- Dacheng Tao
- Stephan Günnemann
- Philip S. Yu
- Mingsheng Long
- Jianmin Wang
In Graph Learning
- Jure Leskovec
- William L. Hamilton
- Stephan Günnemann
- Matthias Fey
- Rex Ying
- Marinka Zitnik
- Michael M. Bronstein
- Aleksandar Bojchevski
Citations in the corpus
- Graph Attention Networks (2017, methodology): “Inductive learning For the inductive learning task, we compare against the four different supervised GraphSAGE inductive methods presented in Hamilton et al. (2017).”
- Stochastic Training of Graph Convolutional Networks (2017, methodology): “To reduce the receptive field size, Hamilton et al. (2017a) propose a neighbor sampling (NS) algorithm.”
- Knowledge Graph Convolutional Networks for Recommender Systems (2019, methodology): “rty of CNN, researchers propose learning a weight matrix for each node degree [6], extracting locally connected regions from graphs [13], or sampling a fixed-size set of neighbors as the support size [7]. Our work can be seen as a non-spectral method for a special type of graphs (i.e., knowledge gra”
- Strategies for Pre-training Graph Neural Networks (2019, methodology): “We also experimented with other popular architectures that are less expressive: GCN (Kipf & Welling, 2016), GAT (Veliˇckovi´c et al., 2019), and GraphSAGE (with mean neighborhood aggregation) (Hamilton et al., 2017b).”
- Graph Convolutional Neural Networks for Web-Scale Recommender Systems (2018, methodology): “Hamilton et al. (2017b) [19] and Bronstein et al. (2017) [6] provide comprehensive surveys of recent advancements.”
- Graph Convolutional Networks for Text Classification (2018, methodology): “Possible solutions to the problem are introducing inductive (Hamilton, Ying, and Leskovec 2017) or fast GCN model (Chen, Ma, and Xiao 2018).”
- Fast Graph Representation Learning with PyTorch Geometric (2019, background): “…(Sen et al., 2008; Bojchevski & Günnemann, 2018), the Coauthor CS/Physics and Amazon Computers/Photo datasets from Shchur et al. (2018), the molecule datasets QM7b (Montavon et al., 2013) and QM9 (Ramakrishnan et al., 2014), and the protein-protein interaction graphs from Hamilton et al. (2017).”
- Compositional Fairness Constraints for Graph Embeddings (2019, methodology): “In more recent years, the task of embedding graph-structured data has received increasing attention from the machine learning and data mining communities (Cai et al., 2018; Hamilton et al., 2017b).”
- Auto-GNN: Neural architecture search of graph neural networks (2019, methodology): “• Activation function: The set of available activation func-tionsinourAGNNislistedasfollows: { Sigmoid , Tanh , ReLU , Linear , Softplus , LeakyReLU , ReLU6 , ELU } Note that a wide-variety of state-of-the-art model fall into the above message-passing based GNN architecture, including Cheby-shev [30”
- On the Binding Problem in Artificial Neural Networks (2020, background): “They have been successfully applied to a wide variety of graph-structured data including social networks (Hamilton et al., 2017), citation networks (Kipf and Welling, 2017), 3D surfaces (Litany et al.”
- Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication (2020, neutral): “A range of relation kernels has been proposed [9,12,21].”
- Relational inductive biases, deep learning, and graph networks (2018, background): “Similarly, Toyer et al. (2017) showed generalization to different sizes of planning problems, and Hamilton et al. (2017) showed generalization to producing useful node embeddings for previously unseen data.”
- Differentiable Physics-informed Graph Networks (2019, background): “One branch of these studies is a spectral approach to aggregate locally connected attributes (Bruna et al., 2014; Defferrard et al., 2016; Kipf & Welling, 2017) and another branch is a spatial approach (or message passing method) (Gilmer et al., 2017; Monti et al., 2017; Hamilton et al., 2017).”
- Position-aware Graph Neural Networks (2019, methodology): “Learning low-dimensional vector representations of nodes in graphs (Hamilton et al., 2017b) has led to advances on tasks such as node classification (Kipf & Welling, 2017), link prediction (Grover & Leskovec, 2016), graph classification (Ying et al., 2018b) and graph generation (You et al., 2018b),…”
- Unsupervised Inductive Whole-Graph Embedding by Preserving Graph Proximity (2019, methodology): “Examples include GCN as in (Kipf & Welling, 2016b), as well as a series of improvements upon GCN, such as GRAPHSAGE (Hamilton et al., 2017), GAT (Velickovic et al., 2018), GIN (Xu et al., 2019), etc. as mentioned in the main paper.”
- Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity (2019, methodology): “A series of neighbor aggregation methods have been proposed with different ways to aggregate neighbor information, e.g. GRAPHSAGE [Hamilton et al., 2017], GAT [Velickovic et al., 2018], GIN [Xu et al., 2019], etc.”
- How Powerful are Graph Neural Networks? (2018, methodology): “Recently, there has been a surge of interest in Graph Neural Network (GNN) approaches for representation learning of graphs (Li et al., 2016; Hamilton et al., 2017a; Kipf & Welling, 2017; Velickovic et al., 2018; Xu et al., 2018).”
- Deep Graph Infomax (2018, methodology): “Therefore, we use the subsampling approach of [8], where a minibatch of nodes is first selected, and then a subgraph centered around each of them is obtained by sampling node neighborhoods with replacement.”