A Survey on Oversmoothing in Graph Neural Networks

By T. Konstantin Rusch, Michael M. Bronstein, Siddhartha Mishra. arXiv.org, 2023

371 citations recorded by the source corpus.

Node features of graph neural networks (GNNs) tend to become more similar with the increase of the network depth. This effect is known as over-smoothing, which we axiomatically define as the exponential convergence of suitable similarity measures on the node features. Our definition unifies previous approaches and gives rise to new quantitative measures of over-smoothing. Moreover, we empirically demonstrate this behavior for several over-smoothing measures on different graphs (small-, medium-, and large-scale). We also review several approaches for mitigating over-smoothing and empirically test their effectiveness on real-world graph datasets. Through illustrative examples, we demonstrate that mitigating over-smoothing is a necessary but not sufficient condition for building deep GNNs that are expressive on a wide range of graph learning tasks. Finally, we extend our definition of over-smoothing to the rapidly emerging field of continuous-time GNNs.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/90dead8a056b848be164c2e5cdadfa2e191c3265

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Heng Ji
2. Jiawei Han
3. Meng Jiang
4. Bowen Jin
5. Chi Han
6. Gang Liu
7. Cynthia Rudin
8. Charu C. Aggarwal

Citations in the corpus

• Large Language Models on Graphs: A Comprehensive Survey (2023, background): “Subsequent work on GNN tackles the issues of over-smoothing [139], over-squashing [140], interpretability [145], and bias [143].”