Relational Deep Learning: A Deep Latent Variable Model for Link Prediction

By Hao Wang, Xingjian Shi, Dit-Yan Yeung. 2017

63 citations recorded by the source corpus.

Link prediction is a fundamental task in such areas as social network analysis, information retrieval, and bioinformatics. Usually link prediction methods use the link structures or node attributes as the sources of information. Recently, the relational topic model (RTM) and its variants have been proposed as hybrid methods that jointly model both sources of information and achieve very promising accuracy. However, the representations (features) learned by them are still not effective enough to represent the nodes (items). To address this problem, we generalize recent advances in deep learning from solely modeling i.i.d. sequences of attributes to jointly modeling graphs and non-i.i.d. sequences of attributes. Specifically, we follow the Bayesian deep learning framework and devise a hierarchical Bayesian model, called relational deep learning (RDL), to jointly model high-dimensional node attributes and link structures with layers of latent variables. Due to the multiple nonlinear transformations in RDL, standard variational inference is not applicable. We propose to utilize the product of Gaussians (PoG) structure in RDL to relate the inferences on different variables and derive a generalized variational inference algorithm for learning the variables and predicting the links. Experiments on three real-world datasets show that RDL works surprisingly well and significantly outperforms the state of the art.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dit-Yan Yeung
2. Hao Wang
3. Seyed Mehran Kazemi
4. David Poole
5. Yu Zhang
6. Qiang Yang
7. Xing Xie
8. Yu Zhang

In Statistical and Nonlinear Physics

1. Dit-Yan Yeung
2. Hao Wang
3. David Poole
4. Seyed Mehran Kazemi

Citations in the corpus

• Towards Bayesian Deep Learning: A Survey (2016, methodology): “ink Prediction. Link prediction has long been a core problem in network analysis and is recently attracting more interest with the new advancements brought by BDL and deep neural networks in general. [120] proposed the first BDL-based model, dubbed relational deep learning (RDL), for link prediction”
• RelNN: A Deep Neural Model for Relational Learning (2017, background): “Examples of these works include path-constrained random walks (e.g., (Lao and Cohen 2010; Lao, Mitchell, and Cohen 2011)), and several hybrid approaches (e.g., (Das et al. 2017; Rockt¨aschel and Riedel 2017; Yang, Yang, and Cohen 2017; Wang, Shi, and Yeung 2017)).”
• A Survey on Bayesian Deep Learning (2016, methodology): “[120] proposed the first BDL-based model, dubbed relational deep learning (RDL), for link prediction.”