Dropout Training of Matrix Factorization and Autoencoder for Link Prediction in Sparse Graphs

By Shuangfei Zhai, Zhongfei Zhang. 2015

38 citations recorded by the source corpus.

Matrix factorization (MF) and Autoencoder (AE) are among the most successful approaches of unsupervised learning. While MF based models have been extensively exploited in the graph modeling and link prediction literature, the AE family has not gained much attention. In this paper we investigate both MF and AE's application to the link prediction problem in sparse graphs. We show the connection between AE and MF from the perspective of multiview learning, and further propose MF+AE: a model training MF and AE jointly with shared parameters. We apply dropout to training both the MF and AE parts, and show that it can significantly prevent overfitting by acting as an adaptive regularization. We conduct experiments on six real world sparse graph datasets, and show that MF+AE consistently outperforms the competing methods, especially on datasets that demonstrate strong non-cohesive structures.

Subfields: Artificial Intelligence, Molecular Biology, Statistical and Nonlinear Physics, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Marinka Zitnik
2. Jure Leskovec
3. Aditya Grover
4. Pietro Morerio
5. Riccardo Volpi
6. William L. Hamilton
7. René Vidal
8. Vittorio Murino

In Molecular Biology

1. Marinka Zitnik
2. Jure Leskovec
3. Aditya Grover
4. William L. Hamilton
5. René Vidal
6. Stephan Günnemann
7. Petar Velickovic
8. Aleksandar Bojchevski

In Statistical and Nonlinear Physics

1. Marinka Zitnik
2. Aditya Grover
3. Jure Leskovec
4. René Vidal
5. Jaewon Yang
6. William L. Hamilton
7. Manuel Gomez-Rodriguez
8. Rok Sosic

In Representation & Self-Supervised Learning

1. Marinka Zitnik
2. Jure Leskovec
3. Aditya Grover
4. William L. Hamilton
5. René Vidal
6. Vittorio Murino
7. Pietro Morerio
8. Jacopo Cavazza

Citations in the corpus

• node2vec: Scalable Feature Learning for Networks (2016, background): “Finally, for both node and edge based prediction tasks, there is a body of recent work for supervised feature learning based on existing and novel graph-specific deep network architectures [15, 16, 17, 31, 39].”
• Predicting multicellular function through multi-layer tissue networks (2017, background): “Finally, there exists recent work for task-dependent feature learning based on graph-specific deep network architectures (Li et al., 2015; Xiaoyi et al., 2014; Wang et al., 2016a; Zhai and Zhang, 2015).”
• Curriculum Dropout (2017, background): “Zhai and Zhang [33] investigate the idea of dropout once applied to matrix factorization.”