Continual Representation Learning for Biometric Identification

By Bo Zhao, Shixiang Tang, Dapeng Chen, Hakan Bilen, Rui Zhao. 2020

50 citations recorded by the source corpus.

With the explosion of digital data in recent years, continuously learning new tasks from a stream of data without forgetting previously acquired knowledge has become increasingly important. In this paper, we propose a new continual learning (CL) setting, namely "continual representation learning", which focuses on learning better representation in a continuous way. We also provide two large-scale multi-step benchmarks for biometric identification, where the visual appearance of different classes are highly relevant. In contrast to requiring the model to recognize more learned classes, we aim to learn feature representation that can be better generalized to not only previously unseen images but also unseen classes/identities. For the new setting, we propose a novel approach that performs the knowledge distillation over a large number of identities by applying the neighbourhood selection and consistency relaxation strategies to improve scalability and flexibility of the continual learning model. We demonstrate that existing CL methods can improve the representation in the new setting, and our method achieves better results than the competitors.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, background): “…when we apply smaller (30) number of epochs during the second phase and compare it to a longer second phase (150 epochs). shorten the second Due to the negligible differences we use the initially chosen value that equals 150 epochs throughout the main paper. phase 0.2 points higher accuracy”
Class-Incremental Instance Segmentation via Multi-Teacher Networks (2021, background): “A variety of incremental learning strategies (Li and Hoiem 2017; Dhar et al. 2019; Zhao et al. 2021) have been explored to prevent models from forgetting previously learned tasks.”
Layerwise Optimization by Gradient Decomposition for Continual Learning (2021, neutral): “This phenomenon, known as catastrophic forgetting, invokes more effective algorithms for continual learning (CL), the goal of which is to learn consecutive tasks without severe performance degradation on previous tasks [5, 30, 34, 38, 44, 43, 57, 57, 50].”
Patch-based Knowledge Distillation for Lifelong Person Re-Identification (2022, methodology): “In order to alleviate catastrophic forgetting without accessing old data, most lifelong ReID methods [46, 56, 69] have adopted a knowledge distillation strategy.”
Learning Continual Compatible Representation for Re-indexing Free Lifelong Person Re-identification (2024, methodology): “In this section, we compared our C 2 R with several state-of-the-art methods on both RFL-ReID and traditional L-ReID tasks, including SPD [28], LwF [11], CRL [40], AKA [18], MEGE [20], PatchKD [27], and CVS [30], where CSV is a continual compatible training method and the others are L-ReID methods.”
Lifelong Person Re-Identification via Adaptive Knowledge Accumulation (2021, methodology): “Note that SFT-T and CRL-T (with additional triplet loss) is not beneficial for the first three training steps, because when the number of training identities is large enough, triplet loss contributes less on performance and even leads to conflict with cross-entropy loss [51].”
Gradient Regularized Contrastive Learning for Continual Domain Adaptation (2020, background): “Continual learning (Prabhu, Torr, and Dokania 2020; Parisi et al. 2019; Zhao et al. 2020) addresses catastrophic forgetting in a sequence of supervised learning tasks.”