Fracking Deep Convolutional Image Descriptors

By Edgar Simo-Serra, Eduard Trulls, Luis Ferraz, Iasonas Kokkinos, Francesc Moreno-Noguer. 2014

71 citations recorded by the source corpus.

In this paper we propose a novel framework for learning local image descriptors in a discriminative manner. For this purpose we explore a siamese architecture of Deep Convolutional Neural Networks (CNN), with a Hinge embedding loss on the L2 distance between descriptors. Since a siamese architecture uses pairs rather than single image patches to train, there exist a large number of positive samples and an exponential number of negative samples. We propose to explore this space with a stochastic sampling of the training set, in combination with an aggressive mining strategy over both the positive and negative samples which we denote as "fracking". We perform a thorough evaluation of the architecture hyper-parameters, and demonstrate large performance gains compared to both standard CNN learning strategies, hand-crafted image descriptors like SIFT, and the state-of-the-art on learned descriptors: up to 2.5x vs SIFT and 1.5x vs the state-of-the-art in terms of the area under the curve (AUC) of the Precision-Recall curve.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Training Region-Based Object Detectors with Online Hard Example Mining (2016, background): “There is recent work [22, 27, 33] concurrent to our own that selects hard examples for training deep networks.”
Recent advances in convolutional neural networks (2015, background): “Online bootstrapping (or hard negative mining [139]) for CNN training has recently gained interest due to its importance for intelligent cognitive systems interacting with dynamically changing environments [58], [140], [141].”
Actions ~ Transformations (2015, methodology): “To model the transformation, we apply a Siamese network architecture in this paper, which is also related to the literature using deep metric learning [11, 9, 37, 12, 54, 16].”
A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection (2017, background): “Other examples of findind hard examples for training include [20, 35, 41].”
Mining on Manifolds: Metric Learning Without Labels (2018, background): “It is common practice, and known to be beneficial [49], to select hard negative examples.”
Fine-Tuning CNN Image Retrieval with No Human Annotation (2017, methodology): “We choose hard negatives [6], [46], that is, non-matching images with the most similar descriptor.”
CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples (2016, methodology): “Following a well-known procedure, we choose hard negatives [44,20], that is, non-matching images with the most similar descriptor.”
Geodesic Convolutional Neural Networks on Riemannian Manifolds (2015, methodology): “For this purpose, we use a siamese network configuration [6, 18, 41], composed of two identical copies of the same GCNN model sharing the same parameterization and fed by pairs of knowingly similar or dissimilar samples, and minimize the siamese loss”
Unsupervised Hard Example Mining from Videos for Improved Object Detection (2018, background): “Hard negative mining has also improved the performance of deep learning based models [47,35,19,46,64,55,33].”