GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning

By Rishabh Tiwari, Krishnateja Killamsetty, Rishabh K. Iyer, Pradeep Shenoy. 2021

183 citations recorded by the source corpus.

Continual learning (CL) aims to develop techniques by which a single model adapts to an increasing number of tasks encountered sequentially, thereby potentially leveraging learnings across tasks in a resource-efficient manner. A major challenge for CL systems is catastrophic forgetting, where earlier tasks are forgotten while learning a new task. To address this, replay-based CL approaches maintain and repeatedly retrain on a small buffer of data selected across encountered tasks. We propose Gradient Coreset Replay (GCR), a novel strategy for replay buffer selection and update using a carefully designed optimization criterion. Specifically, we select and maintain a ‘coreset’ that closely approximates the gradient of all the data seen so far with respect to current model parameters, and discuss key strategies needed for its effective application to the continual learning setting. We show significant gains (2%-4% absolute) over the state-of-the-art in the well-studied offline continual learning setting. Our findings also effectively transfer to online / streaming CL settings, showing up to 5% gains over existing approaches. Finally, we demonstrate the value of supervised contrastive loss for continual learning, which yields a cumulative gain of up to 5% accuracy when combined with our subset selection strategy.

Subfields: Artificial Intelligence, Ocean Engineering, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Adrian Weller
2. Zhen Liu
3. Weiyang Liu
4. Lanqing Hong
5. L. Yu
6. Tianyang Hu
7. Yutong Xie
8. Jianpeng Zhang

In Ocean Engineering

1. Weiyang Liu

In Meta & Continual Learning

1. Lanqing Hong
2. Weiyang Liu

Citations in the corpus

• Continual Learning by Modeling Intra-Class Variation (2022, methodology): “GCR (Tiwari et al., 2022) proposes a gradient-based selection strategy by approximating the gradients of all the data seen so far with respect to current model parameters.”
• Continual Self-Supervised Learning: Towards Universal Multi-Modal Medical Data Representation Learning (2023, background): “In supervised continual learning, a large buffer size usually equates to improved performance due to reduced forgetting [5, 34, 42].”