ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis

By Eu Wern Teh, Terrance Devries, Graham W. Taylor. 2020

196 citations recorded by the source corpus.

We consider the problem of distance metric learning (DML), where the task is to learn an effective similarity measure between images. We revisit ProxyNCA and incorporate several enhancements. We find that low temperature scaling is a performance-critical component and explain why it works. Besides, we also discover that Global Max Pooling works better in general when compared to Global Average Pooling. Additionally, our proposed fast moving proxies also addresses small gradient issue of proxies, and this component synergizes well with low temperature scaling and Global Max Pooling. Our enhanced model, called ProxyNCA++, achieves a 22.9 percentage point average improvement of Recall@1 across four different zero-shot retrieval datasets compared to the original ProxyNCA algorithm. Furthermore, we achieve state-of-the-art results on the CUB200, Cars196, Sop, and InShop datasets, achieving Recall@1 scores of 72.2, 90.1, 81.4, and 90.9, respectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Herv'e J'egou
2. Matthieu Cord
3. Hugo Touvron
4. Matthijs Douze
5. Alexandre Sablayrolles
6. Ludwig Schmidt
7. Quoc V. Le
8. Trevor Darrell

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Jitendra Malik
4. Trevor Darrell
5. Ming-Hsuan Yang
6. Xiaogang Wang
7. Andrea Vedaldi
8. Andrew Zisserman

In Representation & Self-Supervised Learning

1. Hugo Touvron
2. Matthieu Cord
3. Herv'e J'egou
4. Alexandre Sablayrolles
5. Matthijs Douze
6. Björn Ommer
7. Matthieu Guillaumin
8. Jitendra Malik

Citations in the corpus

• Grafit: Learning fine-grained image representations with coarse labels (2020, methodology): “[61], several Distance Metric Learning (DML) approaches like the Magnet loss [44] or ProxyNCA [40, 51] jointly take into account intra- and inter-class variability.”
• Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning (2021, methodology): “To approximate such a setting, current DML benchmarks use single, predefined and fixed data splits of disjoint train and test classes, which are assigned arbitrarily [71, 8, 61, 24, 11, 33, 74, 51, 54, 42, 26, 64, 58].”
• Task-Oriented Multi-User Semantic Communications (2021, background): “While the training algorithms focus on classification based learning [30], metric based learning [31], and unsupervised-based learning [32].”
• Training Vision Transformers for Image Retrieval (2021, unknown): “A first class of deep metric learning methods is based on classification: these approaches represent each category using one (Movshovitz-Attias et al., 2017; Teh et al., 2020; Zhai & Wu, 2018; Boudiaf et al., 2020) or multiple prototypes (Qian et al., 2019).”
• ABO: Dataset and Benchmarks for Real-World 3D Object Understanding (2021, methodology): “Method To evaluate their performance on our multi-view retrieval benchmark, we use the available PyTorch implementations of three recent works that are state-of-theart for their respective approach to deep metric learning: NormSoftmax, (classification-based), ProxyNCA++ (proxy-based) and CGD (triple”