Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks

By Yujun Yan, Milad Hashemi, Kevin Swersky, Yaoqing Yang, Danai Koutra. Industrial Conference on Data Mining, 2021

309 citations recorded by the source corpus.

In node classification tasks, graph convolutional neural networks (GCNs) have demonstrated competitive performance over traditional methods on diverse graph data. However, it is known that the performance of GCNs degrades with increasing number of layers (oversmoothing problem) and recent studies have also shown that GCNs may perform worse in heterophilous graphs, where neighboring nodes tend to belong to different classes (heterophily problem). These two problems are usually viewed as unrelated, and thus are studied independently, often at the graph filter level from a spectral perspective.We are the first to take a unified perspective to jointly explain the oversmoothing and heterophily problems at the node level. Specifically, we profile the nodes via two quantitative metrics: the relative degree of a node (compared to its neighbors) and the node-level heterophily. Our theory shows that the interplay of these two profiling metrics defines three cases of node behaviors, which explain the oversmoothing and heterophily problems jointly and can predict the performance of GCNs. Based on insights from our theory, we show theoretically and empirically the effectiveness of two strategies: structure-based edge correction, which learns corrected edge weights from structural properties (i.e., degrees), and feature-based edge correction, which learns signed edge weights from node features. Compared to other approaches, which tend to handle well either heterophily or oversmoothing, we show that our model, GGCN, which incorporates the two strategies performs well in both problems. We provide a longer version of this paper in [1] and codes on https://github.com/YujunYan/Heterophily_and_oversmoothing.

Subfields: Artificial Intelligence, Social Psychology, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/5b1ce73eee6d80a3897bb03468b067f6583fdba8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

Doina Precup

In Statistical and Nonlinear Physics

Citations in the corpus

A critical look at the evaluation of GNNs under heterophily: are we really making progress? (2023, methodology): “Yan et al. (2022) related learning under heterophily to the problem of oversmoothing and suggested two modiﬁcations to the GCN architecture: degree corrections and signed messages.”
Graph Neural Networks with Heterophily (2020, background): “2020) and explored the theoretical connections of these two problems (Yan et al. 2021).”
Finding Global Homophily in Graph Neural Networks When Meeting Heterophily (2022, methodology): “For example, to jointly study the heterophily and over-smoothing problems, GGCN (Yan et al., 2021) allows for signed messages to be propagated from a node’s neighborhood.”
Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, methodology): “Various GNNs have been proposed to achieve higher performance in low-homophily settings [82, 44, 81, 17, 15, 73, 36, 35].”
Revisiting Heterophily For Graph Neural Networks (2022, background): “Heterophily has received lots of attention recently and an increasing number of models have been put forward to address this problem [46, 29, 32, 8, 45, 44, 33, 16, 24].”