Class-incremental Learning with Pre-allocated Fixed Classifiers

By Federico Pernici, Matteo Bruni, Claudio Baecchi, F. Turchini, A. del Bimbo. 2020

44 citations recorded by the source corpus.

In class-incremental learning, a learning agent faces a stream of data with the goal of learning new classes while not forgetting previous ones. Neural networks are known to suffer under this setting, as they forget previously acquired knowledge. To address this problem, effective methods exploit past data stored in an episodic memory while expanding the final classifier nodes to accommodate the new classes. In this work, we substitute the expanding classifier with a novel fixed classifier in which a number of pre-allocated output nodes are subject to the classification loss right from the beginning of the learning phase. Contrarily to the standard expanding classifier, this allows: (a) the output nodes of future unseen classes to firstly see negative samples since the beginning of learning together with the positive samples that incrementally arrive; (b) to learn features that do not change their geometric configuration as novel classes are incorporated in the learning model. Experiments with public datasets show that the proposed approach is as effective as the expanding classifier while exhibiting novel intriguing properties of the internal feature representation that are otherwise not-existent. Our ablation study on pre-allocating a large number of classes further validates the approach.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Forward Compatible Few-Shot Class-Incremental Learning (2022, methodology): “Note that iCaRL, EEIL, Pre-Allocated RPC, and Rebalancing are traditional class-incremental algorithms.”
Class-Incremental Learning: A Survey (2023, methodology): “Similarly, Neural Architecture Search (NAS) [176] is also adopted to find the optimal structure for each of the sequential tasks [61].”
Regular Polytope Networks (2021, background): “A preliminary exploration of this work was presented in [18] and [19].”
Insights from the Future for Continual Learning (2020, background): “[37], another concurrent work, investigate the notion of future/unseen classes but use negative samples to train the classifier nodes of those classes while we use positive samples through ghost features.”
Deep Class-Incremental Learning: A Survey (2023, background): “RPC [117] claims that the classifiers of all classes can be pre-allocated and fixed.”
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning (2023, background): “A recent study proposes to reserve feature space for novel classes to circumvent their conﬂict with old classes (Zhou et al., 2022a), but an optimal feature-classiﬁer alignment is hard to be guaranteed with learnable classiﬁer (Pernici et al., 2021).”