Large Scale Incremental Learning

By Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, Yun Raymond Fu. 2019

1,535 citations recorded by the source corpus.

Modern machine learning suffers from \textit{catastrophic forgetting} when learning new classes incrementally. The performance dramatically degrades due to the missing data of old classes. Incremental learning methods have been proposed to retain the knowledge acquired from the old classes, by using knowledge distilling and keeping a few exemplars from the old classes. However, these methods struggle to \textbf{scale up to a large number of classes}. We believe this is because of the combination of two factors: (a) the data imbalance between the old and new classes, and (b) the increasing number of visually similar classes. Distinguishing between an increasing number of visually similar classes is particularly challenging, when the training data is unbalanced. We propose a simple and effective method to address this data imbalance issue. We found that the last fully connected layer has a strong bias towards the new classes, and this bias can be corrected by a linear model. With two bias parameters, our method performs remarkably well on two large datasets: ImageNet (1000 classes) and MS-Celeb-1M (10000 classes), outperforming the state-of-the-art algorithms by 11.1\% and 13.2\% respectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Han-Jia Ye
  2. De-chuan Zhan
  3. Bernt Schiele
  4. Jun Zhu
  5. Dacheng Tao
  6. Joost van de Weijer
  7. Mohit Bansal
  8. Yoshua Bengio

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Bernt Schiele
  4. Ming-Hsuan Yang
  5. Trevor Darrell
  6. Xiaogang Wang
  7. Yi Yang
  8. Andrea Vedaldi

In Meta & Continual Learning

  1. Han-Jia Ye
  2. De-chuan Zhan
  3. Da-Wei Zhou
  4. Joost van de Weijer
  5. Liyuan Wang
  6. Jun Zhu
  7. Xingxing Zhang
  8. Bing Liu

Citations in the corpus