ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection

By Huiping Zhuang, Zhenyu Weng, Renchunzi Xie, Kar-Ann Toh, Zhiping Lin. 2022

91 citations recorded by the source corpus.

Class-incremental learning (CIL) learns a classification model with training data of different classes arising progressively. Existing CIL either suffers from serious accuracy loss due to catastrophic forgetting, or invades data privacy by revisiting used exemplars. Inspired by linear learning formulations, we propose an analytic class-incremental learning (ACIL) with absolute memorization of past knowledge while avoiding breaching of data privacy (i.e., without storing historical data). The absolute memorization is demonstrated in the sense that class-incremental learning using ACIL given present data would give identical results to that from its joint-learning counterpart which consumes both present and historical samples. This equality is theoretically validated. Data privacy is ensured since no historical data are involved during the learning process. Empirical validations demonstrate ACIL's competitive accuracy performance with near-identical results for various incremental task settings (e.g., 5-50 phases). This also allows ACIL to outperform the state-of-the-art methods for large-phase scenarios (e.g., 25 and 50 phases).

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Qiwen Wang
5. Ziwei Liu
6. Zhiyuan Qi
7. Hai-Long Sun
8. Chao Du

Citations in the corpus

• Locality Sensitive Sparse Encoding for Learning World Models Online (2024, methodology): “…we can observe the error for Losse-FTL decreases, verifying our design that the high dimensional sparse feature brings more fitting capacity while maintaining the same computational cost with our sparse update. this topic focusing on class-incremental learning (Zhuang et al., 2022; 2023; 2024).”
• GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task (2023, background): “This coincides well with the incremental scenarios and it is introduced to CIL realm in the analytic class-incremental learning (ACIL) [36] which has state-of-the-art performance via a frozen backbone and linear classifier trained by analytic learning.”
• Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning (2024, background): “Class-Incremental Learning (CIL) : requires a learning system to continually absorb new class knowledge without forgetting existing ones [13, 14, 20, 22, 38, 57, 59, 74, 81, 82], which can be roughly divided into several categories.”
• Class-Incremental Learning: A Survey (2023, methodology): “To this end, exemplar-free CIL [50], [150], [223], [224], [225] should be conducted to enable the model to be adapted without the help of exemplars.”