Towards Backward-Compatible Representation Learning

By Yantao Shen, Yuanjun Xiong, Wei Xia, Stefano Soatto. 2020

100 citations recorded by the source corpus.

We propose a way to learn visual features that are compatible with previously computed ones even when they have different dimensions and are learned via different neural network architectures and loss functions. Compatible means that, if such features are used to compare images, then ``new'' features can be compared directly to ``old'' features, so they can be used interchangeably. This enables visual search systems to bypass computing new features for all previously seen images when updating the embedding models, a process known as backfilling. Backward compatibility is critical to quickly deploy new embedding models that leverage ever-growing large-scale training datasets and improvements in deep learning architectures and training methods. We propose a framework to train embedding models, called backward-compatible training (BCT), as a first step towards backward compatible representation learning. In experiments on learning embeddings for face recognition, models trained with BCT successfully achieve backward compatibility without sacrificing accuracy, thus enabling backfill-free model updates of visual embeddings.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Forward Compatible Few-Shot Class-Incremental Learning (2022, background): “Recent work focuses on improving model’s backward compatibility [7, 28, 35], while we are the first to address the model’s forward compatibility in FSCIL.”
Metric Compatible Training for Online Backfilling in Large-Scale Retrieval (2023, unknown): “Following the previous works [17, 20, 26], we measure mean average precision (mAP) and cumulative matching characteristics (CMC) 3 .”
General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference (2020, background): “In a real-world settings it may be necessary to store extracted features for later use, such as when new tasks are introduced that require “backﬁlling” classiﬁcations for older content (Shen et al., 2020).”
Class-incremental Learning with Pre-allocated Fixed Classifiers (2020, methodology): “Freezing the classifier is also the key strategy used in [42] to learn visual features that are compatible with features computed with different CNN models.”
LQF: Linear Quadratic Fine-Tuning (2020, methodology): “We leave various applications of LQF to future work, including its use in (i) uncertainty quantification, (ii) backward compatibility [44], (iii) leave-one-out cross validation without a validation set [9], (iv) active/online learning [45], (v) counterfactual analysis in the first order, (vi) contin”
Learning Continual Compatible Representation for Re-indexing Free Lifelong Person Re-identification (2024, methodology): “Existing CT methods can be categorized into two branches: backward CT [16, 24, 39] and forward CT methods [21, 30, 45].”
Asymmetric metric learning for knowledge transfer (2020, methodology): “Our asymmetric test scenario is equivalent to that of prior studies [27, 60], but with different motivation and settings.”
Learning Compatible Embeddings (2021, methodology): “Few efforts, R(3)AN [4], RBT [44] (for CMC) and BCT [38] (for CT), are proposed and reach comparable results.”
Boundary-aware Backward-Compatible Representation via Adversarial Learning in Image Retrieval (2023, methodology): “The convergence trend of BCT [18], LCE [14],”
Forward Compatible Training for Large-Scale Embedding Retrieval Systems (2021, unknown): “Following [5], we randomly sample 50 images for each subject and use this as both the query and gallery sets.”
When Embedding Models Meet: Procrustes Bounds and Applications (2025, unknown): “A practical difficulty arises when the query model is upgraded but document embeddings cannot be recomputed, either because the raw documents are not available (Morris et al., 2023; Huang et al., 2024), or recomputation is too costly (Shen et al., 2020; Arora et al., 2020).”
Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models (2025, unknown): “While re-indexing was found to be effective in image retrieval (Shen et al., 2020), we show that in CDR, re-indexing of old task documents does not improve performance significantly and performs worse compared to FT on initial tasks like MS MARCO and NQ.”