FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning

By Grégoire Petit, Adrian-Stefan Popescu, Hugo Schindler, David Picard, Bertrand Delezoide. 2022

201 citations recorded by the source corpus.

Exemplar-free class-incremental learning is very challenging due to the negative effect of catastrophic forgetting. A balance between stability and plasticity of the incremental process is needed in order to obtain good accuracy for past as well as new classes. Existing exemplar-free class-incremental methods focus either on successive fine tuning of the model, thus favoring plasticity, or on using a feature extractor fixed after the initial incremental state, thus favoring stability. We introduce a method which combines a fixed feature extractor and a pseudo-features generator to improve the stability-plasticity balance. The generator uses a simple yet effective geometric translation of new class features to create representations of past classes, made of pseudo-features. The translation of features only requires the storage of the centroid representations of past classes to produce their pseudo-features. Actual features of new classes and pseudo-features of past classes are fed into a linear classifier which is trained incrementally to discriminate between all classes. The incremental process is much faster with the proposed method compared to mainstream ones which update the entire deep model. Experiments are performed with three challenging datasets, and different incremental settings. A comparison with ten existing methods shows that our method outperforms the others in most cases. FeTrIL code is available at https://github.com/GregoirePetit/FeTrIL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “Other methods exploit the class-wise statistics of feature representations to obtain a balanced classifier, such as by imposing the representations to be transferable and invariant (SPB [453], IL2A [509], SSRE [512], FeTrIL [333]), or compensating explicitly the representation shifts (SDC [484], RER”
Deep Class-Incremental Learning: A Survey (2023, methodology): “To this end, exemplar-free CIL [57], [58], [138], [139] should be conducted to enable the model to be adapted without the help of exemplars.”
Elastic Feature Consolidation for Cold Start Exemplar-free Incremental Learning (2024, methodology): “In Cold Start scenarios, we observe once again that freezing the feature extractor does not provide enough plasticity for FeTrIL to learn the new tasks effectively, resulting in a significant decrease (more than 8”
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning (2024, methodology): “This includes LwF [24], PASS [55], SSRE [56], FeTRIL [35] and Fe-CAM [11].”
Class-Incremental Learning: A Survey (2023, methodology): “Under such circumstances, algorithms should be designed to learn without exemplars [49] or using privacy-friendly strategies like feature replay [50].”
Divide and not forget: Ensemble of selectively trained experts in Continual Learning (2024, methodology): “We additionally compare SEED to the best competitor - FeTrIL (with Resnet32) in a low parameters regime.”
Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models (2025, unknown): “In our setting, we consider that during the training of task t we have no access to any data from previous tasks (also known as exemplar-free continual learning (Smith et al., 2023; Petit et al., 2023; Goswami et al., 2023; Magistri et al., 2024)).”