Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay

By Fan Zhou, Chengtai Cao. 2021

192 citations recorded by the source corpus.

Graph Neural Networks (GNNs) have recently received significant research attention due to their superior performance on a variety of graph-related learning tasks. Most of the current works focus on either static or dynamic graph settings, addressing a single particular task, e.g., node/graph classification, link prediction. In this work, we investigate the question: can GNNs be applied to continuously learning a sequence of tasks? Towards that, we explore the Continual Graph Learning (CGL) paradigm and present the Experience Replay based framework ER-GNN for CGL to alleviate the catastrophic forgetting problem in existing GNNs. ER-GNN stores knowledge from previous tasks as experiences and replays them when learning new tasks to mitigate the catastrophic forgetting issue. We propose three experience node selection strategies: mean of feature, coverage maximization, and influence maximization, to guide the process of selecting experience nodes. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our ER-GNN and shed light on the incremental graph (non-Euclidean) structure learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ruocheng Guo
2. Kaize Ding
3. Jaewoo Kang
4. Dacheng Tao
5. Zhen Tan
6. Seoyoon Kim
7. Dongjin Song
8. Seongjun Yun

In Computer Vision and Pattern Recognition

1. Jaewoo Kang
2. Dacheng Tao
3. Zhen Tan
4. Seoyoon Kim
5. Dongjin Song
6. Ruocheng Guo
7. Seongjun Yun
8. Xikun Zhang

Citations in the corpus

• Lifelong Graph Learning (2020, neutral): “For example, several exemplar selection methods are tested on ER-GNN [58].”
• DyGRAIN: An Incremental Learning Framework for Dynamic Graphs (2022, methodology): “Recently, a few incremental learning meth-ods for graph-structured data have emerged, but they investigate a static setting where aims to improve class learning (e.g., class-incremental setting) and focus to overcome only the catastrophic forgetting problem [Zhou and Cao, 2021; Liu et al. , 2021].”
• Graph Few-shot Class-incremental Learning (2021, methodology): “• Prototypical Networks on graphs: As discussed in Section 4.2, • State-of-the-art Graph Few-shot learning methods: Meta-GNN • Continual learning methods on graph: ER-GNN [49], and • Few-shot Class-incremental Learning: CEC [47].”
• CGLB: Benchmark Tasks for Continual Graph Learning (2022, methodology): “…topology for previous tasks, parametric isolation approaches ( e.g. , Hierarchical Pro-totype Networks (HPNs) [63]) that select different parameters for different tasks, and memory replay methods ( e.g. , Experience Replay Graph Neural Network (ER-GNN) [64]) that stores representative nodes.”