Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking

By Aleksandar Bojchevski, Stephan Günnemann. 2017

780 citations recorded by the source corpus.

Methods that learn representations of nodes in a graph play a critical role in network analysis since they enable many downstream learning tasks. We propose Graph2Gauss - an approach that can efficiently learn versatile node embeddings on large scale (attributed) graphs that show strong performance on tasks such as link prediction and node classification. Unlike most approaches that represent nodes as point vectors in a low-dimensional continuous space, we embed each node as a Gaussian distribution, allowing us to capture uncertainty about the representation. Furthermore, we propose an unsupervised method that handles inductive learning scenarios and is applicable to different types of graphs: plain/attributed, directed/undirected. By leveraging both the network structure and the associated node attributes, we are able to generalize to unseen nodes without additional training. To learn the embeddings we adopt a personalized ranking formulation w.r.t. the node distances that exploits the natural ordering of the nodes imposed by the network structure. Experiments on real world networks demonstrate the high performance of our approach, outperforming state-of-the-art network embedding methods on several different tasks. Additionally, we demonstrate the benefits of modeling uncertainty - by analyzing it we can estimate neighborhood diversity and detect the intrinsic latent dimensionality of a graph.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Representation & Self-Supervised Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Representation & Self-Supervised Learning

In Graph Learning

Citations in the corpus

Deep Graph Infomax (2018, methodology): “…ways to score a representation, but in the graph literature the most common techniques use classiﬁcation (Perozzi et al., 2014; Grover & Leskovec, 2016; Kipf & Welling, 2016b; Hamilton et al., 2017b), though other scoring functions are used (Duran & Niepert, 2017; Bojchevski & G¨unnemann, 2018).”
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation (2019, methodology): “…(Raffel et al., 2019; Lan et al., 2020) to PLMs. Knowledge-Enhanced PLMs Recently, many works have investigated how to incorporate knowledge into PLMs. MTB (Baldini Soares et al., 2019) takes a straightforward “matching the blank” pretraining objective to help the relation classification task.”
NetGAN: Generating Graphs via Random Walks (2018, methodology): “For the large CORA dataset and its commonly used subset of machine learning papers denoted with CORA-ML we use the same preprocessing as in Bojchevski & Günnemann (2018).”
All You Need Is Low (Rank): Defending Against Adversarial Attacks on Graphs (2020, methodology): “Pr ( X ≥ σ 210 ) is 0.0013, 0.0019, and 0.0037 on CiteSeer, Cora-ML, and PoliticalBlogs, respectively.”
Graph Neural Networks in Recommender Systems: A Survey (2020, neutral): “, word embedding [140], document embedding [41, 111], and network/graph embedding [9, 53, 159].”
Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks (2021, methodology): “We evaluate the performance of our GGCN model and existing GNNs in node classiﬁcation on various real-world datasets (Tang et al., 2009; Rozemberczki et al., 2019; Sen et al., 2008; Namata et al., 2012; Bojchevski & Günne-mann, 2018; Shchur et al., 2018).”
Graph Random Neural Networks for Semi-Supervised Learning on Graphs (2020, neutral): “Cora-Full is proposed in [4].”
Large-Scale Representation Learning on Graphs via Bootstrapping (2021, positive): “We perform additional experiments on 4 commonly used small datasets, (Cora, CiteSeer, PubMed, and DBLP) (Sen et al., 2008; Bojchevski & Günnemann, 2017) and show that BGRL ’s bootstrapping mechanism still performs well in the low-data regime, even attaining a new state of the art performance on two…”
Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs (2020, background): “9 Real datasets & setup We now evaluate the performance of our model and existing GNNs on a variety of real-world datasets [35, 29, 30, 22, 4, 31] with edge homophily ratio h ranging from strong heterophily to strong homophily, going beyond the traditional Cora, Pubmed and Citeseer graphs that have ”