Online Continual Learning from Imbalanced Data

By Aristotelis Chrysakis, Marie-Francine Moens. 2020

176 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “More advanced strategies prioritize the replay of those training samples that are informative (InfoRS [398]), diversified in parameter gradients (GSS [17]) or previously-learned knowledge (ODDL [473]), balanced in class labels (CBRS [75], GDumb [340], CoPE [86]), and beneficial to latent decision bo”
Improving Task-free Continual Learning by Distributionally Robust Memory Evolution (2022, background): “Task-free continual learning (He et al., 2019; Zeno et al., 2019; Aljundi et al., 2019b; Chrysakis & Moens, 2020) instead focuses on the more general case where the data distribution could change arbitrarily without explicit task splits.”
Information-theoretic Online Memory Selection for Continual Learning (2022, neutral): “…Reserovir sampling (RS) (Vitter, 1985), weighted reservoir sampling with the output-space Hessian (WRS), class-balanced reservoir sampling (CBRS) (Chrysakis & Moens, 2020), greedy gradient sample selection (GSS) (Aljundi et al., 2019b), feature-space clustering (FSS-Clust) (Aljundi et al.,…”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “These statistics are then used as complementary information to handle the data imbalance problem.”
Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021, background): “The closest works (Chrysakis and Moens, 2020; Kim et al., 2020) maintain only the balanced label distribution in the memory while our”
An EM Framework for Online Incremental Learning of Semantic Segmentation (2021, positive): “We then extend the classbalanced reservoir sampling (CBRS) [11] to the incremental learning of semantic segmentation by taking into account the multiple classes in each image.”
Rehearsal revealed: The limits and merits of revisiting samples in continual learning (2021, methodology): “Random retrieval has been widely adopted [7, 6, 9, 8] while an alternative evaluates the potentially highest increase in the loss [2].”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…Aljundi, Lin, Goujaud, & Bengio, 2019; He, Mao, Shao, & Zhu, 2020; Zhao, Xiao, Gan, Zhang, & Xia, 2020; Kurle, Cseke, Klushyn, van der Smagt, & Günnemann, 2020; Chrysakis & Moens, 2020; Kim, Jeong, & Kim, 2020; Tao, Chang, Hong, Wei, & Gong, 2020; Douil-lard, Cord, Ollion, Robert, & Valle, 2020).”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “What differentiate various memory-based methods are the memory retrieval strategy (line 3) [17, 85], model update (line 4) [31, 13] and memory update strategy (line 5) [61, 63, 18].”
Online Class-Incremental Continual Learning with Adversarial Shapley Value (2020, methodology): “Online Class-Incremental Learning Following the recent CL literature (Aljundi et al. 2019a,b; Lee et al. 2020; Chrysakis and Moens 2020), we consider the online supervised class-incremental learning setting where a model needs to learn new classes continually from an online data stream (each sample…”
Coresets via Bilevel Optimization for Continual Learning and Streaming (2020, methodology): “To the best of our knowledge, the replay-based approach to streaming has been tackled by [2, 27, 11], which we compare against experimentally.”
The Effectiveness of Memory Replay in Large Scale Continual Learning (2020, neutral): “…new update rules using replay samples (Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2018), while others focus on new sampling strategies to ﬁll the replay buffer (Rebufﬁ et al., 2017; Chaudhry et al., 2020; Aljundi et al., 2019; Hayes et al., 2019; Chrysakis & Moens, 2020; Borsos et al., 2020).”
InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning (2024, background): “…of the subspace M ⊥ t in different Transformer layers of ViT-B/16. ﬁnd We can that the dimension of the subspace M ⊥ t in different Transformer layers of ViT-B/16 is always much larger than zero, which means the space for learning the new task always exists throughout the whole learning process.”
Saliency Guided Experience Packing for Replay in Continual Learning (2021, background): “Continual learning algorithms, especially OCL methods, have achieved SOTA performance using experience replay [19, 5, 7].”
Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness (2021, background): “To our best knowledge, the replay-based approach to imbalanced streaming setting of continual learning has been only considered in [5, 17, 32].”
Machine Unlearning for Random Forests (2020, background): “This makes them well-suited to continual learning settings with streaming data (Chrysakis & Moens, 2020; Knoblauch et al., 2020).”
CGLB: Benchmark Tasks for Continual Graph Learning (2022, methodology): “Better designs of memory replay are also explored recently [6, 9] since it is simple yet effective.”