Gradient based sample selection for online continual learning

By Rahaf Aljundi, Min Lin, Baptiste Goujaud, Yoshua Bengio. 2019

1,049 citations recorded by the source corpus.

A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal. Previous works often depend on task boundary and i.i.d. assumptions to properly select samples for the replay buffer. In this work, we formulate sample selection as a constraint reduction problem based on the constrained optimization view of continual learning. The goal is to select a fixed subset of constraints that best approximate the feasible region defined by the original constraints. We show that it is equivalent to maximizing the diversity of samples in the replay buffer with parameters gradient as the feature. We further develop a greedy alternative that is cheap and efficient. The advantage of the proposed method is demonstrated by comparing to other alternatives under the continual learning setting. Further comparisons are made against state of the art methods that rely on task boundaries which show comparable or even better results for our method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Dacheng Tao
  2. Yoshua Bengio
  3. Jun Zhu
  4. Bing Liu
  5. Sergey Levine
  6. Han-Jia Ye
  7. Mohit Bansal
  8. De-chuan Zhan

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Trevor Darrell
  5. Bernt Schiele
  6. Xiaogang Wang
  7. Andrea Vedaldi
  8. Andrew Zisserman

In Optimization & Training

  1. Frederik Benzing
  2. Tinne Tuytelaars
  3. Alexander Rainer Tassilo Gepperth
  4. Joost van de Weijer
  5. Rahaf Aljundi
  6. Fei Ye
  7. Adrian G. Bors
  8. Tyler L. Hayes

In Meta & Continual Learning

  1. Liyuan Wang
  2. Xingxing Zhang
  3. Jun Zhu
  4. Bing Liu
  5. Tinne Tuytelaars
  6. Joost van de Weijer
  7. Hang Su
  8. Fei Ye

Citations in the corpus