Label Mapping Neural Networks with Response Consolidation for Class Incremental Learning

By Xu Zhang, Yang Yao, Baile Xu, Le Mao, Shen Furao, Jian Zhao, Qingwei Lin. 2019

Class incremental learning refers to a special multi-class classification task, in which the number of classes is not fixed but is increasing with the continual arrival of new data. Existing researches mainly focused on solving catastrophic forgetting problem in class incremental learning. To this end, however, these models still require the old classes cached in the auxiliary data structure or models, which is inefficient in space or time. In this paper, it is the first time to discuss the difficulty without support of old classes in class incremental learning, which is called as softmax suppression problem. To address these challenges, we develop a new model named Label Mapping with Response Consolidation (LMRC), which need not access the old classes anymore. We propose the Label Mapping algorithm combined with the multi-head neural network for mitigating the softmax suppression problem, and propose the Response Consolidation method to overcome the catastrophic forgetting problem. Experimental results on the benchmark datasets show that our proposed method achieves much better performance compared to the related methods in different scenarios.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Da-Wei Zhou
2. Xi Xiao
3. Bowen Zhao
4. Bin Zhang
5. Shutao Xia
6. Guojun Gan
7. Jiangpeng He
8. Fei Mi

Citations in the corpus

• Maintaining Discrimination and Fairness in Class Incremental Learning (2019, methodology): “Thus, the new model trained by the vanilla method generally predicts objects as new classes [26, 32, 38].”