Learnability and Algorithm for Continual Learning

By Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Bin Liu. 2023

37 citations recorded by the source corpus.

This paper studies the challenging continual learning (CL) setting of Class Incremental Learning (CIL). CIL learns a sequence of tasks consisting of disjoint sets of concepts or classes. At any time, a single model is built that can be applied to predict/classify test instances of any classes learned thus far without providing any task related information for each test instance. Although many techniques have been proposed for CIL, they are mostly empirical. It has been shown recently that a strong CIL system needs a strong within-task prediction (WP) and a strong out-of-distribution (OOD) detection for each task. However, it is still not known whether CIL is actually learnable. This paper shows that CIL is learnable. Based on the theory, a new CIL algorithm is also proposed. Experimental results demonstrate its effectiveness.

Subfields: Artificial Intelligence, Education, Meta & Continual Learning, Statistical Learning Theory

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “This is closely related to the out-of-distribution (OOD) detection, where the predictive uncertainty should be low for in-distribution data and high for OOD data [74], [75], [76], [77].”