Continual Learning Based on OOD Detection and Task Masking

By Gyuhak Kim, Sepideh Esmaeilpour, Changnan Xiao, Bing Liu. 2022

29 citations recorded by the source corpus.

Existing continual learning techniques focus on either task incremental learning (TIL) or class incremental learning (CIL) problem, but not both. CIL and TIL differ mainly in that the task-id is provided for each test sample during testing for TIL, but not provided for CIL. Continual learning methods intended for one problem have limitations on the other problem. This paper proposes a novel unified approach based on out-of-distribution (OOD) detection and task masking, called CLOM, to solve both problems. The key novelty is that each task is trained as an OOD detection model rather than a traditional supervised learning model, and a task mask is trained to protect each task to prevent forgetting. Our evaluation shows that CLOM outperforms existing state-of-the-art baselines by large margins. The average TIL/CIL accuracy of CLOM over six experiments is 87.6/67.9% while that of the best baselines is only 84.4/55.0%. The code of our system is available at https://github.com/k-gyuhak/CLOM.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bin Liu
2. Gyuhak Kim
3. Tatsuya Konishi
4. Changnan Xiao
5. Zixuan Ke
6. Bing Liu
7. Neil Zhenqiang Gong
8. Jinyuan Jia

In Computer Vision and Pattern Recognition

1. Bin Liu
2. Gyuhak Kim
3. Zixuan Ke
4. Changnan Xiao
5. Tatsuya Konishi
6. Bing Liu

Citations in the corpus

• Open-World Continual Learning: Unifying Novelty Detection and Continual Learning (2023, result): “For the forgetting rate results in the TIL setting, refer to our earlier workshop paper [28].”
• Online Continual Learning through Mutual Information Maximization (2022, unknown): “CLOM (Kim et al., 2022) combines parameter isolation and out-of-distribution detection for continual learning.”
• A Multi-Head Model for Continual Learning via Out-of-Distribution Replay (2022, methodology): “Our recent work CLOM (Kim et al., 2022) uses contrastive learning and data augmentation to build an OOD classifier for each task, but it is not a replay method and is weaker than MORE.”