Continual Learning on Noisy Data Streams via Self-Purified Replay
By Chris Dongjoo Kim, Jinseo Jeong, Sang-chul Moon, Gunhee Kim. 2021
58 citations recorded by the source corpus.
Continually learning in the real world must overcome many challenges, among which noisy labels are a common and inevitable issue. In this work, we present a replay-based continual learning framework that simultaneously addresses both catastrophic forgetting and noisy labels for the first time. Our solution is based on two observations; (i) forgetting can be mitigated even with noisy labels via self-supervised learning, and (ii) the purity of the replay buffer is crucial. Building on this regard, we propose two key components of our method: (i) a self-supervised replay technique named Self-Replay which can circumvent erroneous training signals arising from noisy labeled data, and (ii) the Self-Centered filter that maintains a purified replay buffer via centrality-based stochastic graph ensembles. The empirical results on MNIST, CIFAR-10, CIFAR-100, and WebVision with real-world noise demonstrate that our framework can maintain a highly pure replay buffer amidst noisy streamed data while greatly outperforming the combinations of the state-of-the-art continual learning and noisy label learning methods.
Subfields: Artificial Intelligence, Civil and Structural Engineering, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hang Su
- Jun Zhu
- Liyuan Wang
- Xingxing Zhang
- Lorenzo Bonicelli
- Concetto Spampinato
- Matteo Boschini
- Simone Calderara
In Civil and Structural Engineering
In Meta & Continual Learning
- Xingxing Zhang
- Liyuan Wang
- Hang Su
- Jun Zhu
- Lorenzo Bonicelli
- Concetto Spampinato
- Simone Calderara
- Matteo Boschini
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “To cope with noisy labeled incremental data, SPR [216] obtains self-supervised representations to purify a memory buffer and then performs supervised finetuning.”
- Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (2022, background): “5, RSV, GBS and RM have small amount of training time but poor accuracy, while SPR takes too much time to be utilized in online setup.”
- Representation Compensation Networks for Continual Semantic Segmentation (2022, background): “To solve this problem, many work [4, 5,11,47,77] propose to review knowledge by rehearsal-based mechanism.”
- Effects of Auxiliary Knowledge on Continual Learning (2022, background): “tasks [32], [33], [7], [34], or by making use of semi-supervised learning techniques to learn general features [35], [36], [37].”