Graph Information Bottleneck

By Tailin Wu, Hongyu Ren, Pan Li, Jure Leskovec. Neural Information Processing Systems, 2020

350 citations recorded by the source corpus.

Representation learning of graph-structured data is challenging because both graph structure and node features carry important information. Graph Neural Networks (GNNs) provide an expressive way to fuse information from network structure and node features. However, GNNs are prone to adversarial attacks. Here we introduce Graph Information Bottleneck (GIB), an information-theoretic principle that optimally balances expressiveness and robustness of the learned representation of graph-structured data. Inheriting from the general Information Bottleneck (IB), GIB aims to learn the minimal sufficient representation for a given task by maximizing the mutual information between the representation and the target, and simultaneously constraining the mutual information between the representation and the input data. Different from the general IB, GIB regularizes the structural as well as the feature information. We design two sampling algorithms for structural regularization and instantiate the GIB principle with two new models: GIB-Cat and GIB-Bern, and demonstrate the benefits by evaluating the resilience to adversarial attacks. We show that our proposed models are more robust than state-of-the-art graph defense models. GIB-based models empirically achieve up to 31% improvement with adversarial perturbation of the graph structure as well as node features.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/2fce1ef37391cd685fc5459e1cbfcb8490b85242

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Siqi Miao
2. Miaoyuan Liu
3. Jennifer Neville
4. Cong Hao
5. Zenan Li
6. Pan Li
7. Wentao Zhao
8. David Paul Wipf

Citations in the corpus

• Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism (2022, methodology): “Inspired by the graph information bottleneck (GIB) principle (Wu et al., 2020; Yu et al., 2021), we propose to use information constraint instead to select label-relevant subgraphs, i.”
• NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification (2023, background): “, [22, 42], considering weighting the edges in input graph for enhancing the expressiveness.”
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2021, methodology): “Recently, the information bottleneck has applied to learn graph representations [47,48].”