SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

By Qi Qian, Lei Shang, Baigui Sun, Juhua Hu, Hao Li, Rong Jin. IEEE International Conference on Computer Vision, 2019

432 citations recorded by the source corpus.

Distance metric learning (DML) is to learn the embeddings where examples from the same class are closer than examples from different classes. It can be cast as an optimization problem with triplet constraints. Due to the vast number of triplet constraints, a sampling strategy is essential for DML. With the tremendous success of deep learning in classifications, it has been applied for DML. When learning embeddings with deep neural networks (DNNs), only a mini-batch of data is available at each iteration. The set of triplet constraints has to be sampled within the mini-batch. Since a mini-batch cannot capture the neighbors in the original set well, it makes the learned embeddings sub-optimal. On the contrary, optimizing SoftMax loss, which is a classification loss, with DNN shows a superior performance in certain DML tasks. It inspires us to investigate the formulation of SoftMax. Our analysis shows that SoftMax loss is equivalent to a smoothed triplet loss where each class has a single center. In real-world data, one class can contain several local clusters rather than a single one, e.g., birds of different poses. Therefore, we propose the SoftTriple loss to extend the SoftMax loss with multiple centers for each class. Compared with conventional deep metric learning algorithms, optimizing SoftTriple loss can learn the embeddings without the sampling phase by mildly increasing the size of the last fully connected layer. Experiments on the benchmark fine-grained data sets demonstrate the effectiveness of the proposed loss function.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6556cdf36a72c9842dbfb146df84b3e7d633e8df

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning (2020, methodology): “arity Classier (LSC) with NCA loss. Finally, we add LSC-CE: our classier with multi-mode but with a simple cross-entropy loss instead of our modied NCA loss. This version brings to mind SoftTriple [32] and Innited Mixture Prototypes [2], used in the dierent context of few-shot learning. The latter o”
Revisiting Training Strategies and Generalization Performance in Deep Metric Learning (2020, methodology): “Moreover, proxy-based approaches address this issue by approximating class distributions using only few virtual representatives (Movshovitz-Attias et al., 2017; Qian et al., 2019).”
Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning (2021, background): “More recent work has tackled sampling complexity through the usage of proxy-representations utilized as sample stand-ins during training, following a NCA [16] objective [41, 26, 64], leveraging softmax-style training through class-proxies [8, 73] or simulating intraclass structures [46].”
Training Vision Transformers for Image Retrieval (2021, neutral): “A ﬁrst class of deep metric learning methods is based on classiﬁcation: these approaches represent each category using one (Movshovitz-Attias et al., 2017; Teh et al., 2020; Zhai & Wu, 2018; Boudiaf et al., 2020) or multiple prototypes (Qian et al., 2019).”
SphereFace Revived: Unifying Hyperspherical Face Recognition (2021, background): “The development of deep FR is also closely related to deep metric learning [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28].”
A Metric Learning Reality Check (2020, background): “Softmax) [58,31,72] 2017 Classification CosFace [57,59] 2018 Classification ArcFace [11] 2019 Classification FastAP [3] 2019 Embedding Signal to Noise Ratio Contrastive (SNR) [70] 2019 Embedding MultiSimilarity (MS) [62] 2019 Embedding MS+Miner [62] 2019 Embedding SoftTriple [41] 2019 Classification”
Small-Task Incremental Learning (2020, methodology): “This version brings to mind SoftTriple [27], used in the different context of few-shot learning.”
Sub-center ArcFace: Boosting Face Recognition by Large-Scale Noisy Web Faces (2020, methodology): “The proposed sub-center ArcFace is different from Softtriple [22] in the following aspects: – Softtriple shows improvement in ﬁne-grained retrieval by employing multi-centers.”