Deep Metric Learning with Hierarchical Triplet Loss

By Weifeng Ge, Weilin Huang, Dengke Dong, Matthew R. Scott. European Conference on Computer Vision, 2018

465 citations recorded by the source corpus.

We present a novel hierarchical triplet loss (HTL) capable of automatically collecting informative training samples (triplets) via a defined hierarchical tree that encodes global context information. This allows us to cope with the main limitation of random sampling in training a conventional triplet loss, which is a central issue for deep metric learning. Our main contributions are two-fold. (i) we construct a hierarchical class-level tree where neighboring classes are merged recursively. The hierarchical structure naturally captures the intrinsic data distribution over the whole dataset. (ii) we formulate the problem of triplet collection by introducing a new violate margin, which is computed dynamically based on the designed hierarchical tree. This allows it to automatically select meaningful hard samples with the guide of global context. It encourages the model to learn more discriminative features from visual similar classes, leading to faster convergence and better performance. Our method is evaluated on the tasks of image retrieval and face recognition, where it outperforms the standard triplet loss substantially by 1%–18%, and achieves new state-of-the-art performance on a number of benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ece216e4140258302263c0401b474b691b4134d4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Metric Learning Reality Check (2020, methodology): “Other sophisticated training methods include HTL [14], ABE [25], MIC [43], and DCES [46].”
Classification is a Strong Baseline for Deep Metric Learning (2018, background): “Hierarchical Triplet Loss [24] proposed that by merging similar classes dynamically during training into a hierarchical tree, more informative samples can be drawn from such a structure and the loss also provides global context for training.”
Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning (2021, methodology): “However, due to the exponentially increased complexity of tuple sampling spaces, these methods are usually also combined with tuple sampling objectives, relying on predefined or learned heuristics to avoid training over tuples that are too easy or too hard [57, 72] or reducing tuple redudancy encoun”
Revisiting Training Strategies and Generalization Performance in Deep Metric Learning (2020, methodology): “To address the computational complexity of tuple-based meth- ods1, different sampling strategies have been introduced (Schroff et al., 2015; Wu et al., 2017; Ge, 2018; Roth et al., 2020).”
Contrastive Learning with Hard Negative Samples (2020, background): “As opposed to the contrastive representation learning literature, selection strategies for negative samples have been thoroughly studied in (deep) metric learning (Schroff et al., 2015; Song et al., 2016; Harwood et al., 2017; Wu et al., 2017; Ge, 2018; Suh et al., 2019).”
Circle Loss: A Unified Perspective of Pair Similarity Optimization (2020, methodology): “On these datasets, majority methods [19, 18, 3, 20, 13, 34] adopt the encouraged setting of learning with pair-wise labels.”
Deep Metric Learning: A Survey (2019, methodology): “The authors in [72] proposed a new metric loss that is based on Figure 6.”
Proxy Anchor Loss for Deep Metric Learning (2020, unknown): “1 HTL [7] BN 57.”
Deep Anomaly Detection for Generalized Face Anti-Spoofing (2019, background): “Neverthe-less, it has been successfully addressed in many recent approaches [15, 31, 35]. where { a i , p i , n i } sub-indexes are the anchor, the positive and the negative samples for each triplet within the batch, respectively.”
ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis (2020, background): “In addition to contrastive and triplet loss, there is a long line of work which proposes new loss functions, such as angular loss [31], histogram loss [26], margin-based loss [33], and hierarchical triplet loss [6].”
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning (2020, background): “[21] leverage a contrastive loss to enforce the positive samples to stay in a close neighborhood and negative samples far apart [26, 27, 28, 29].”
When and why vision-language models behave like bags-of-words, and what to do about it? (2022, background): “Negative mining and contrastive learning : Using hard negatives has been successful in improving representation learning (Harwood et al., 2017; Wu et al., 2017; Ge, 2018).”
Imbalance Problems in Object Detection: A Review (2019, methodology): “In order to model such similarities, the methods generally employ pairs [162], [163] or triplets [164], [165], [166] during training.”