Principal Neighbourhood Aggregation for Graph Nets

By Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Lio', Petar Velickovic. 2020

873 citations recorded by the source corpus.

Graph Neural Networks (GNNs) have been shown to be effective models for different predictive tasks on graph-structured data. Recent work on their expressive power has focused on isomorphism tasks and countable feature spaces. We extend this theoretical framework to include continuous features - which occur regularly in real-world input domains and within the hidden layers of GNNs - and we demonstrate the requirement for multiple aggregation functions in this context. Accordingly, we propose Principal Neighbourhood Aggregation (PNA), a novel architecture combining multiple aggregators with degree-scalers (which generalize the sum aggregator). Finally, we compare the capacity of different models to capture and exploit the graph structure via a novel benchmark containing multiple tasks taken from classical graph theory, alongside existing benchmarks from real-world domains, all of which demonstrate the strength of our model. With this work, we hope to steer some of the GNN research towards new aggregation methods which we believe are essential in the search for powerful and robust models.

Subfields: Computational Theory and Mathematics, Computer Networks and Communications, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Computer Networks and Communications

In Signal Processing

Citations in the corpus

Auto-GNN: Neural architecture search of graph neural networks (2019, background): “A variety of GNNs based on spatial graph convolutions has been developed, including GNN models with the different aggregation mechanisms (Hamilton et al., 2017; Corso et al., 2020), and the different attentions (Vaswani et al.”
ETA Prediction with Graph Neural Networks in Google Maps (2021, methodology): “carefully tuned combinations of aggregators [5]) that proved particularly important for maintaining stability of the GNN in this regime and making it production-ready.”
How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks (2020, methodology): “If we can identify h from domain knowledge, then the model only needs to learn g (Corso et al., 2020; Lample & Charton, 2020).”
Do Transformers Really Perform Bad for Graph Representation? (2021, neutral): “…0.002 GAT [50] 531,345 0.384 ± 0.007 GCN [26] 505,079 0.367 ± 0.011 GatedGCN-PE [4] 505,011 0.214 ± 0.006 MPNN (sum) [15] 480,805 0.145 ± 0.007 PNA [10] 387,155 0.142 ± 0.010 GT [13] 588,929 0.226 ± 0.014 SAN [28] 508, 577 0.139 ± 0.006 Graphormer SLIM 489,321 0.122 ± 0.006 the other pre-trained…”
Combinatorial optimization and reasoning with graph neural networks (2021, background): “Such ndings have been independently veri ed [62, 105, 22] and contradict the more common advice to use the sum-aggregator [143].”
Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction (2021, methodology): “PNA significantly improves over simple counterparts, which highlights the importance of learning more powerful AGGREGATE functions.”
Weisfeiler and Lehman Go Cellular: CW Networks (2021, background): “[20] Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković.”
Graph Neural Networks with Learnable Structural and Positional Representations (2021, methodology): “…= σ ( η̂`ij )∑ j′∈N (i) σ ( η̂`ij′ ) + , (21) η̂`ij = B ` 1h ` i +B ` 2h ` j +B ` 3e ` ij , (22) where h`i , e ` ij , p ` i , η ` ij , η̂ ` ij ∈ Rd, A`1, A`2 ∈ Rd×2d and B`1, B`2, B`3, C`1, C`2 ∈ Rd×d. C.1.2 PNA-LSPE PNA (Corso et al., 2020) is a sparse MP-GNN model which uses a combination of…”
Pre-training Molecular Graph Representation with 3D Geometry (2021, methodology): “Second, we acknowledge there are a lot of advanced 3D [23, 45, 58, 78] and 2D [13, 16, 29, 54, 100, 102] representation methods.”
Recipe for a General, Powerful, Scalable Graph Transformer (2022, background): “We compare GPS against a set of popular message-passing neural networks (GCN, GIN, GatedGCN, PNA, etc.), graph transformers (SAN, Graphormer, etc.), and other recent graph neural networks with SOTA results (CIN, CRaWL, GIN-AK+, ExpC).”
Rethinking Graph Transformers with Spectral Attention (2021, methodology): “[15, 21], against GCN [25], GraphSage [18], GIN [39], GAT [37], GatedGCN [5], PNA [11], and DGN [4].”
Benchmarking Graph Neural Networks (2023, methodology): “This dataset is crucial to benchmark the robustness of a GNN to predict specific or overall of all the 6 properties, as these may share subroutines such as graph traversals, despite the tasks being different graph properties (Corso et al., 2020).”
Representing Long-Range Context for Graph Neural Networks with Global Attention (2022, neutral): “…from [18] 0.1399 ± 0.0017 0 .1507 ± 0.0018 GIN-Virtual [36] 0.1439 ± 0.0026 0 .1581 ± 0.0020 GCN-Virtual [18] 0.1461 ± 0.0013 0 .1595 ± 0.0018 PNA [8] 0.1453 ± 0.0025 0 .1570 ± 0.0032 DAGNN (SOTA) [27] 0.1607 ± 0.0040 0 .1751 ± 0.0049 Transformer [28] 0.1546 ± 0.0018 0 .1670 ± 0.0015 GraphTrans…”
Artificial Neural Networks and Machine Learning – ICANN 2021: 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part III (2021, background): “Performance results of our model on molecular property prediction datasets against: DGN [2], PNA [7], GIN, GCN, and DeeperGCN [28] and DeeperGCN/GIN-FLAG [26].”
MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design (2022, methodology): “PNA [34] considers various aggregation methods during message passing.”
Structure-Aware Transformer for Graph Representation Learning (2022, methodology): “We consider k-subtree and k-subgraph SAT equipped with different GNN extractors, including GCN, GIN, GraphSAGE and PNA.”
Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks (2021, background): “This result relies on MPSN’s ability of mapping multi-sets of colours injectively and the fact that neural networks that can learns such functions for multi-sets of bounded size exist (Xu et al., 2019b; Corso et al., 2020).”