Network Adaptation Strategies for Learning New Classes without Forgetting the Original Ones
By Hagai Taitelbaum, Gal Chechik, Jacob Goldberger. 2019
We address the problem of adding new classes to an existing classifier without hurting the original classes, when no access is allowed to any sample from the original classes. This problem arises frequently since models are often shared without their training data, due to privacy and data ownership concerns. We propose an easy-to-use approach that modifies the original classifier by retraining a suitable subset of layers using a linearly-tuned, knowledge-distillation regularization. The set of layers that is tuned depends on the number of new added classes and the number of original classes.We evaluate the proposed method on two standard datasets, first in a language-identification task, then in an image classification setup. In both cases, the method achieves classification accuracy that is almost as good as that obtained by a system trained using unrestricted samples from both the original and new classes.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Idan Achituve
- Gal Chechik
- Aviv Navon
- Ethan Fetaya
- Yochai Yemini
- Han-Jia Ye
- Pedro H. O. Pinheiro
- Song Tian
Citations in the corpus
- GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning (2021, background): “(2017) limit the update of the parameters when encountering new data based on the fisher matrix ; (ii) Architectural based methods suggest network architectures that are resilient to the catastrophic forgetting issue and can accommodate new tasks (Mallya et al., 2018; Mallya & Lazebnik, 2018; Yoon e”