Revisiting Semi-Supervised Learning with Graph Embeddings
By Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov. 2016
2,546 citations recorded by the source corpus.
We present a semi-supervised learning framework based on graph embeddings. Given a graph between instances, we train an embedding for each instance to jointly predict the class label and the neighborhood context in the graph. We develop both transductive and inductive variants of our method. In the transductive variant of our method, the class labels are determined by both the learned embeddings and input feature vectors, while in the inductive variant, the embeddings are defined as a parametric function of the feature vectors, so predictions can be made on instances not seen during training. On a large and diverse set of benchmark tasks, including text classification, distantly supervised entity extraction, and entity classification, we show improved performance over many of the existing models.
Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Quoc V. Le
- Percy Liang
- Jianfeng Gao
- Ruslan Salakhutdinov
- Sergey Levine
- Luke S. Zettlemoyer
In Representation & Self-Supervised Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Yoshua Bengio
- Philip S. Yu
- Jian Tang
- Aleksandar Bojchevski
- Bryan Perozzi
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Aleksandar Bojchevski
- Matthias Fey
- Michael M. Bronstein
- Jian Tang
- Marinka Zitnik
Citations in the corpus
- Semi-Supervised Classification with Graph Convolutional Networks (2016, methodology): “We choose the same dataset splits as in Yang et al. (2016) with an additional validation set of 500 labeled examples for hyperparameter optimization (dropout rate for all layers, L2 regularization factor for the first GCN layer and number of hidden units).”
- Inductive Representation Learning on Large Graphs (2017, neutral): “[40], which is an inductive, embeddingbased approach to semi-supervised learning.”
- Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs (2016, methodology): “Finally, in the network analysis community, several works constructed graph embeddings [36, 53, 13, 20, 60] methods inspired by the Word2Vec technique [33].”
- Semi-Supervised QA with Generative Domain-Adaptive Nets (2017, background): “The problem is challenging because conventional manifold-based semi-supervised learning algorithms (Zhu and Ghahramani, 2002; Yang et al., 2016a) cannot be straightforwardly applied.”
- Deep Graph Infomax (2018, methodology): “…50 runs of training (followed by logistic regression), and reuse the metrics already reported in Kipf & Welling (2016a) for the performance of DeepWalk and GCN, as well as Label Propagation (LP) (Zhu et al., 2003) and Planetoid (Yang et al., 2016)—a representative supervised random walk method.”
- Representation Learning on Graphs: Methods and Applications (2017, methodology): “However, many node embedding algorithms—especially the neighborhood aggregation approaches presented in Section 2.3.2—can also incorporate task-specific supervision [28, 34, 52, 59].”
- Graph Normalizing Flows (2019, methodology): “Transductive learning tasks consist of semi-supervised document classification in citation networks (Cora and Pubmed datasets), where we test our model with the author’s dataset splits [28], as well as 1% train split for a fair comparison against [18].”
- Graphite: Iterative Generative Modeling of Graphs (2018, methodology): “We report the baseline results for SemiEmb [53], DeepWalk [35], ICA [27] and Planetoid [54] as specified in [23].”
- Predict then Propagate: Graph Neural Networks meet Personalized PageRank (2018, background): “…great success on many important graph problems such as link prediction (Grover & Leskovec, 2016; Bojchevski et al., 2018), graph classification (Duvenaud et al., 2015; Niepert et al., 2016; Gilmer et al., 2017) and semi-supervised node classification (Yang et al., 2016; Kipf & Welling, 2017).”
- Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning (2018, methodology): “We compare our methods with several state-of-the-art methods, including GCN with validation (GCN+V); GCN without validation (GCN-V); GCN with Chebyshev filter (Cheby) (Kipf and Welling 2017); label propagation using ParWalks (LP) (Wu et al. 2012); Planetoid (Yang, Cohen, and Salakhutdinov 2016); Deep”
- Simplifying Graph Convolutional Networks (2019, methodology): “Graph embedding methods (Weston et al., 2008; Perozzi et al., 2014; Yang et al., 2016; Velikovi et al., 2019) aim at representing nodes as feature vectors in a high-dimensional continuous space.”
- Pitfalls of Graph Neural Network Evaluation (2018, methodology): “We run the 4 models on the datasets and respective splits from [Yang et al., 2016].”
- Attention-based Graph Neural Network for Semi-supervised Learning (2018, methodology): “The classification accuracy of all the baseline methods are collected from [46, 27, 38, 31, 11, 41].”
- Good Semi-supervised Learning That Requires a Bad GAN (2017, methodology): “Traditional graph-based methods [2, 26] were extended to deep neural networks [22, 23, 8], which involves applying convolutional neural networks [10] and feature learning techniques to graphs so that the underlying manifold structure can be exploited.”
- GRAM: Graph-based Attention Model for Healthcare Representation Learning (2016, background): “Graph convolutional approaches (Yang et al., 2016; Kipf and Welling, 2016) also focus on learning the vertex representations to mainly perform vertex classification.”