Graph Neural Networks with Learnable Structural and Positional Representations

By Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson. International Conference on Learning Representations, 2021

497 citations recorded by the source corpus.

Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce Positional Encoding (PE) of nodes, and inject it into the input layer, like in Transformers. Possible graph PE are Laplacian eigenvectors. In this work, we propose to decouple structural and positional representations to make easy for the network to learn these two essential properties. We introduce a novel generic architecture which we call LSPE (Learnable Structural and Positional Encodings). We investigate several sparse and fully-connected (Transformer-like) GNNs, and observe a performance increase for molecular datasets, from 1.79% up to 64.14% when considering learnable PE for both GNN classes.

Subfields: Artificial Intelligence, Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/454304628bf10f02aba1c2cfc95891e94d09208e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Vijay Prakash Dwivedi
2. Anh Tuan Luu
3. Xavier Bresson
4. Thomas Laurent
5. Dominique Beaini
6. Yoshua Bengio
7. Chaitanya K. Joshi
8. Karsten M. Borgwardt

In Materials Chemistry

1. Vijay Prakash Dwivedi
2. Dominique Beaini
3. Anh Tuan Luu
4. Xavier Bresson
5. Thomas Laurent
6. Yoshua Bengio
7. Karsten M. Borgwardt
8. Guy Wolf

Citations in the corpus

• Recipe for a General, Powerful, Scalable Graph Transformer (2022, methodology): “Some works also propose dedicated networks to learn the PE/SE from an initial encoding [34, 15, 36, 8].”
• Benchmarking Graph Neural Networks (2023, background): “, 2022), (iv) Random-walk PE (Dwivedi et al., 2022), and (v) PEs for Graph Transformers (Dwivedi and Bresson, 2021; Kreuzer et al.”
• Structure-Aware Transformer for Graph Representation Learning (2022, methodology): “We first train a two-layer vanilla Transformer model using RWPE.”