Information-theoretic Online Memory Selection for Continual Learning

By Daniele Calandriello, Huiyi Hu, Ang Li, Michalis K. Titsias. 2022

64 citations recorded by the source corpus.

A challenging problem in task-free continual learning is the online selection of a representative replay memory from data streams. In this work, we investigate the online memory selection problem from an information-theoretic perspective. To gather the most information, we propose the \textit{surprise} and the \textit{learnability} criteria to pick informative points and to avoid outliers. We present a Bayesian model to compute the criteria efficiently by exploiting rank-one matrix structures. We demonstrate that these criteria encourage selecting informative points in a greedy algorithm for online memory selection. Furthermore, by identifying the importance of \textit{the timing to update the memory}, we introduce a stochastic information-theoretic reservoir sampler (InfoRS), which conducts sampling among selective points with high information. Compared to reservoir sampling, InfoRS demonstrates improved robustness against data imbalance. Finally, empirical performances over continual learning benchmarks manifest its efficiency and efficacy.

Subfields: Artificial Intelligence, Meta & Continual Learning, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jun Zhu
2. Hang Su
3. Liyuan Wang
4. Xingxing Zhang
5. Bernard Ghanem
6. Philip H. S. Torr
7. Puneet Kumar Dokania
8. Ser-Nam Lim

In Meta & Continual Learning

1. Liyuan Wang
2. Xingxing Zhang
3. Bernard Ghanem
4. Hang Su
5. Jun Zhu
6. Philip H. S. Torr
7. René Vidal
8. Paris V. Giampouras

In Information Theory

1. Philip H. S. Torr
2. René Vidal
3. Puneet Kumar Dokania
4. Jun Zhu
5. Ser-Nam Lim

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “More advanced strategies prioritize the replay of those training samples that are informative [273], diversified in parameter gradients [21], balanced in class labels [159], [274], and beneficial to latent decision boundaries [134].”
• The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “We believe that selecting representative samples from each task or in a streaming setting can still be beneficial (Borsos et al., 2020; Sun et al., 2022) if the datasets contain some out-of-distribution data.”
• Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “Another kind of approaches for continual learning focuses on the memory system, which usually employs a small memory buffer [3, 5, 6, 28, 36, 44, 59] for storing some past data or trains a generator to provide the replay samples when learning new tasks [38, 43, 46, 47, 58, 57, 52].”
• Computationally Budgeted Continual Learning: What Does Matter? (2023, methodology): “The results in Figure 6 show that the top distillation methods, such as MSE, perform only slightly worse compared to Naive ( 54 .”