Pre-training Graph Neural Networks with Kernels

By Nicoló Navarin, Dinh Van Tran, Alessandro Sperduti. 2018

30 citations recorded by the source corpus.

Many machine learning techniques have been proposed in the last few years to process data represented in graph-structured form. Graphs can be used to model several scenarios, from molecules and materials to RNA secondary structures. Several kernel functions have been defined on graphs that coupled with kernelized learning algorithms, have shown state-of-the-art performances on many tasks. Recently, several definitions of Neural Networks for Graph (GNNs) have been proposed, but their accuracy is not yet satisfying. In this paper, we propose a task-independent pre-training methodology that allows a GNN to learn the representation induced by state-of-the-art graph kernels. Then, the supervised learning phase will fine-tune this representation for the task at hand. The proposed technique is agnostic on the adopted GNN architecture and kernel function, and shows consistent improvements in the predictive performance of GNNs in our preliminary experimental results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Materials Chemistry, Representation & Self-Supervised Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jure Leskovec
2. Marinka Zitnik
3. William L. Hamilton
4. Percy Liang
5. Weihua Hu
6. Bryan Perozzi
7. Bowen Liu
8. Jian Tang

In Computer Vision and Pattern Recognition

1. Jure Leskovec
2. Bryan Perozzi
3. Marinka Zitnik
4. Percy Liang
5. Joseph Gomes
6. Emmanuel Müller
7. John Palowitch
8. Vijay S. Pande

In Materials Chemistry

1. Jure Leskovec
2. Marinka Zitnik
3. Bowen Liu
4. Weihua Hu
5. Joseph Gomes
6. Percy Liang
7. Vijay S. Pande
8. Jian Tang

In Representation & Self-Supervised Learning

1. Jure Leskovec
2. Marinka Zitnik
3. William L. Hamilton
4. Weihua Hu
5. Bryan Perozzi
6. Percy Liang
7. Bowen Liu
8. Vijay S. Pande

In Graph Learning

1. Jure Leskovec
2. Marinka Zitnik
3. William L. Hamilton
4. Weihua Hu
5. Percy Liang
6. Bowen Liu
7. Bryan Perozzi
8. Jian Tang

Citations in the corpus

• Strategies for Pre-training Graph Neural Networks (2019, background): “Examples of such tasks include modeling the graph edit distance (Bai et al., 2019) or predicting graph structure similarity (Navarin et al., 2018).”
• Pre-training Graph Neural Networks (2019, background): “…two options possible for graph-level pre-training: learn composable embeddings either (1) by making predictions on domain-specific auxiliary supervised tasks, or, (2) by performing self-supervised prediction on graph-level structure, e.g. , graph edit distance [3] or graph structure similarity [33].”
• Graph Clustering with Graph Neural Networks (2020, methodology): “A very similar approach was introduced independently in [27] in the context of pre-training GNNs for producing representations of graphs, which was first tackled in [36].”