Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimizing Global Loss Functions

By B. V. Kumar, Gustavo Carneiro, Ian David Reid. 2015

320 citations recorded by the source corpus.

Recent innovations in training deep convolutional neural network (ConvNet) models have motivated the design of new methods to automatically learn local image descriptors. The latest deep ConvNets proposed for this task consist of a siamese network that is trained by penalising misclassification of pairs of local image patches. Current results from machine learning show that replacing this siamese by a triplet network can improve the classification accuracy in several problems, but this has yet to be demonstrated for local image descriptor learning. Moreover, current siamese and triplet networks have been trained with stochastic gradient descent that computes the gradient from individual pairs or triplets of local image patches, which can make them prone to overfitting. In this paper, we first propose the use of triplet networks for the problem of local image descriptor learning. Furthermore, we also propose the use of a global loss that minimises the overall classification error in the training set, which can improve the generalisation capability of the model. Using the UBC benchmark dataset for comparing local image descriptors, we show that the triplet network produces a more accurate embedding than the siamese network in terms of the UBC dataset errors. Moreover, we also demonstrate that a combination of the triplet and global losses produces the best embedding in the field, using this triplet network. Finally, we also show that the use of the central-surround siamese network trained with the global loss produces the best result of the field on the UBC dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Learning Spread-Out Local Feature Descriptors (2017, methodology): “[8] propose to use a global loss to separate the distance distributions of the matching pairs and nonmatching pairs.”
Few-Shot Adversarial Domain Adaptation (2017, background): “Siamese networks [7] work well for subspace learning and have worked very well with deep convolutional neural networks [9, 53, 30, 61].”
Deep Anomaly Detection with Outlier Exposure (2018, background): “The FPRN metric (Liu et al., 2018; Kumar et al., 2016; Balntas et al., 2016) is the probability that an in-distribution example (negative) raises a false alarm when N% of anomalous examples (positive) are detected, so a lower FPRN is better.”
Unified Deep Supervised Domain Adaptation and Generalization (2017, background): “Among algorithms for subspace learning, Siamese networks [11] work well for different tasks [14, 55, 35, 63, 9].”
Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks (2017, methodology): “We employ a global loss function that, according to [8], is robust to outliers and provides better regularisation.”
UvA-DARE (Digital Academic Repository) Self-Supervised Video Representation Learning With Odd-One-Out Networks (unknown): “[26], training of such triplet networks is not straightforward with the need to estimate stable gradients for tripletbased losses.”
A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, methodology): “A siamese based loss function is used in [164] by Kumar et al.”
Time-Contrastive Networks: Self-Supervised Learning from Multi-view Observation (2017, background): “Prior work in this area has studied unsupervised learning as a way of enabling supervised learning from small labeled datasets [7], image retrieval [18], and a variety of other tasks [28, 35, 12, 27, 20, 10].”
Deep Metric Learning with Hierarchical Triplet Loss (2018, background): “…understanding tasks, including images or object retrieval [26,30,34], single-shot object classiﬁcation [30,34,32], key-point descriptor learning [12,24], face veriﬁcation [22,20], person re-identiﬁcation [30,23], object tracking [29] and etc. Recently, there is a number of widely-used loss…”
Smart Mining for Deep Metric Learning (2017, methodology): “ets enable faster learning of the embedding. 5. Conclusion From the results in Tables1-2, we see that Triplet + FANNG + Global + Adaptive signiﬁcantly outperforms the current state of the art methods [10,15] in terms of clustering and recall performances. Furthermore, it is worth Figure 6. a) Triple”
Mining on Manifolds: Metric Learning Without Labels (2018, background): “Of course, given the positive and negative pools, there are many more possibilities in sampling positives and negatives and forming losses that are functions of more than two or three items [29, 36, 5, 27, 59, 67].”
Learning Social Image Embedding with Deep Multimodal Attention Networks (2017, background): “Therefore, fewer parameters and less data are required to train it [8, 27, 28].”
R2D2: Repeatable and Reliable Detector and Descriptor (2019, unknown): “triplet loss [6, 3, 21, 55, 29, 61]).”
Hard-Aware Deeply Cascaded Embedding (2016, background): “…various tasks , such as visual product search [6, 18, 23, 27, 35], face recognition [21, 32, 41, 4], local image descriptor learning [9, 2, 12, 24], person/vehicle reidentification [22, 7, 43, 17], zero-shot image classification [19, 45, 5], fine-grained image classification [8, 40, 44] and…”
Metric learning with spectral graph convolutions on brain connectivity networks (2018, methodology): “We used a siamese graph convolutional neural network that employed the polynomial filters formulated in Defferrard et al. (2016) and a global loss function that, according to Kumar et al. (2016), is robust to outliers and provides better regularisation.”
Working hard to know your neighbor's margins: Local descriptor learning loss (2017, background): “esults on multiscale patch sampling or so called “center-surrounding” architecture for two reasons. First, architectural choices are beyond the scope of current paper. Second, it was already shown in [24, 31] that “center-surrounding” consistently improves results on Brown dataset for different desc”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, background): “Training either takes one example at a time, positive or negative, and adapts the similarity [37], [38], [40], [41], or takes at each step both a positive and a negative example, and maximizes the difference between the similarities, hence aiming at making the two patches from the positive pair more”