Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries

By Jihwan Bang, Hyun-woo Koh, Seulki Park, Hwanjun Song, Jung-Woo Ha, Jonghyun Choi. 2022

61 citations recorded by the source corpus.

Learning under a continuously changing data distribution with incorrect labels is a desirable real-world problem yet challenging. A large body of continual learning (CL) methods, however, assumes data streams with clean labels, and online learning scenarios under noisy data streams are yet underexplored. We consider a more practical CL task setup of an online learning from blurry data stream with corrupted labels, where existing CL methods struggle. To address the task, we first argue the importance of both diversity and purity of examples in the episodic memory of continual learning models. To balance diversity and purity in the episodic memory, we propose a novel strategy to manage and use the memory by a unified approach of label noise aware diverse sampling and robust learning with semi-supervised learning. Our empirical validations on four real-world or synthetic noise datasets (CI-FAR10 and 100, mini-WebVision, and Food-101N) exhibit that our method significantly outperforms prior arts in this realistic and challenging continual learning scenario. Code and data splits are available in https://github.com/clovaai/puridiver.

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Hang Su
5. Qiwen Wang
6. Jun Zhu
7. Ziwei Liu
8. Liyuan Wang

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Qiwen Wang
8. Jun Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Some recent work has focused on various combinations of these scenarios [29], [49], [224], [275], [299], [462], as a way to simulate the complexity of the real world.”
• Coherent Temporal Synthesis for Incremental Action Segmentation (2024, unknown): “The performance of an incremental learning model is evaluated over all seen classes, i.e ., ), but when class overlap occurs, the task is referred to as blurry class-incremental learning (Blurry CIL) [4,5,22].”
• Deep Class-Incremental Learning: A Survey (2023, background): “When Yb ∩ Yb′ 6= ∅, the setting is called blurry class-incremental learning (Blurry CIL) [34], [35], [36].”