Variational Graph Auto-Encoders
By Thomas Kipf, Max Welling. 2016
4,397 citations recorded by the source corpus.
We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE). This model makes use of latent variables and is capable of learning interpretable latent representations for undirected graphs. We demonstrate this model using a graph convolutional network (GCN) encoder and a simple inner product decoder. Our model achieves competitive results on a link prediction task in citation networks. In contrast to most existing models for unsupervised learning on graph-structured data and link prediction, our model can naturally incorporate node features, which significantly improves predictive performance on a number of benchmark datasets.
Subfields: Artificial Intelligence, Molecular Biology, Statistical and Nonlinear Physics, Generative Models, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Jure Leskovec
- Mohit Bansal
- Percy Liang
- Sergey Levine
- William L. Hamilton
- Jianfeng Gao
- Philip S. Yu
In Molecular Biology
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Rex Ying
- Aleksandar Bojchevski
- Marinka Zitnik
- Jian Tang
- Jiaxuan You
In Statistical and Nonlinear Physics
- Jure Leskovec
- William L. Hamilton
- Stephan Günnemann
- Shirui Pan
- Chengqi Zhang
- Bernhard Schölkopf
- Rex Ying
- Stefano Ermon
In Generative Models
- Stefano Ermon
- William L. Hamilton
- Arash Vahdat
- Max Welling
- Diederik P. Kingma
- Yoshua Bengio
- Jun Zhu
- Yang Song
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Matthias Fey
- Rex Ying
- Aleksandar Bojchevski
- Marinka Zitnik
- Jian Tang
Citations in the corpus
- Inductive Representation Learning on Large Graphs (2017, neutral): “So far, graph convolutional networks (GCNs) have only been applied in the transductive setting with fixed graphs [17, 18].”
- Stochastic Training of Graph Convolutional Networks (2017, methodology): “Generalization of GCN to other tasks can be found in Kipf & Welling (2016); Berg et al. (2017); Schlichtkrull et al. (2017) and Hamilton et al. (2017b).”
- Strategies for Pre-training Graph Neural Networks (2019, methodology): “We also experimented with other popular architectures that are less expressive: GCN (Kipf & Welling, 2016), GAT (Veliˇckovi´c et al., 2019), and GraphSAGE (with mean neighborhood aggregation) (Hamilton et al., 2017b).”
- E(n) Equivariant Graph Neural Networks (2021, methodology): “Similarly to the work of (Kipf & Welling, 2016b) further extended by section 3.3 in (Liu et al., 2019), our graph auto-encoder z = q ( G ) embeds a graph G into a set of latent nodes z = { z 1 , . . . z M } ∈ R M × n , where M is the number of nodes and n the embedding size per node.”
- Compositional Fairness Constraints for Graph Embeddings (2019, methodology): “…factorization techniques (Ng et al., 2001), with recent years witnessing a surge in methods that rely on random-walk based objectives (Grover & Leskovec, 2016; Perozzi et al., 2014), deep au-toencoders (Wang et al., 2016), and graph neural networks (Hamilton et al., 2017a; Kipf & Welling, 2016).”
- Position-aware Graph Neural Networks (2019, methodology): “Note that in the context of link prediction task, our implementation of GCN is equivalent to GAE (Kipf & Welling, 2016).”
- Unsupervised Inductive Whole-Graph Embedding by Preserving Graph Proximity (2019, methodology): “For each graph, UGRAPHEMB first generates a set of node embeddings using neighbor aggregation methods based on Graph Convolutional Networks (GCN) (Kipf & Welling, 2016a), which are permutation-invariant and inductive.”
- Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity (2019, methodology): “, 2016; Kipf and Welling, 2016a], such as VGAE [Kipf and Welling, 2016b], satisfy the desired permutation-invariance property.”
- Deep Graph Infomax (2018, methodology): “…deviation) on the test nodes of our method after 50 runs of training (followed by logistic regression), and reuse the metrics already reported in Kipf & Welling (2016a) for the performance of DeepWalk and GCN, as well as Label Propagation (LP) (Zhu et al., 2003) and Planetoid (Yang et al.,…”
- Representation Learning on Graphs: Methods and Applications (2017, methodology): “For example, the decoder might predict the existence of edges between nodes, given their embeddings [1, 35], or it might predict the community that a node belongs to in the graph [28, 34] (Figure 3).”
- Set2Graph: Learning Graphs From Sets (2020, methodology): “Edge prediction is done via outer-product [14].”
- Graph Normalizing Flows (2019, methodology): “Note that in contrast to the GraphVAE [12], which generates a single vector to model the entire graph, we instead embed a set of nodes in a graph jointly, but each node will be mapped to its own embedding vector.”
- Graphite: Iterative Generative Modeling of Graphs (2018, methodology): “We evaluated Graphite-VAE and Graphite-AE against the following baselines: Spectral Clustering (SC) [45], DeepWalk [35], node2vec [13], and GAE/VGAE [22].”
- GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models (2018, methodology): “Building on these approaches a number of deep learning models for generating graphs have been proposed (Kipf & Welling, 2016; Grover et al., 2017; Simonovsky & Ko-modakis, 2018; Li et al., 2018).”
- FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling (2018, methodology): “Specifically, all labels of the training examples are used for training, as opposed to only a small portion in the semi-supervised setting (Kipf & Welling, 2016a).”
- Adversarially Regularized Graph Autoencoder (2018, methodology): “• VGAE [Kipf and Welling, 2016]: is a variational graph autoencoder approach for graph embedding with both topological and content information.”
- Modeling Relational Data with Graph Convolutional Networks (2017, background): “2Note that this represents a simplification of the message passing neural network proposed in (Gilmer et al. 2017) that suffices to include the aforementioned models as special cases.”
- Self-Supervised Learning: Generative or Contrastive (2020, neutral): “Due to the uniqueness of graph-structured data, the objective of VGAE is to reconstruct the adjacency matrix of the graph by measuring node proximity.”