HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors

By Vassileios Balntas, Karel Lenc, Andrea Vedaldi, Krystian Mikolajczyk. Computer Vision and Pattern Recognition, 2017

875 citations recorded by the source corpus.

In this paper, we propose a novel benchmark for evaluating local image descriptors. We demonstrate that the existing datasets and evaluation protocols do not specify unambiguously all aspects of evaluation, leading to ambiguities and inconsistencies in results reported in the literature. Furthermore, these datasets are nearly saturated due to the recent improvements in local descriptors obtained by learning them from large annotated datasets. Therefore, we introduce a new large dataset suitable for training and testing modern descriptors, together with strictly defined evaluation protocols in several tasks such as matching, retrieval and classification. This allows for more realistic, and thus more reliable comparisons in different application scenarios. We evaluate the performance of several state-of-the-art descriptors and analyse their properties. We show that a simple normalisation of traditional hand-crafted descriptors can boost their performance to the level of deep learning based descriptors within a realistic benchmarks evaluation.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/96112084b619243c4b49d990ab60021ab25286b2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

R2D2: Repeatable and Reliable Detector and Descriptor (2019, methodology): “We evaluate our method in the 116 full image sequences of the HPatches dataset [2].”
ASLFeat: Learning Local Features of Accurate Shape and Localization (2020, methodology): “On HPatches dataset [1], three standard metrics are used: 1) Keypoint repeatability ( %Rep. ) , a.k.a. the ratio of possible matches and the minimum number of keypoints in the shared view.”
SOSNet: Second Order Similarity Regularization for Local Descriptor Learning (2019, methodology): “(10) by using 90K randomly selected classes from the HPatches dataset.”
SuperPoint: Self-Supervised Interest Point Detection and Description (2017, methodology): “In our experiments we train SuperPoint on the MSCOCO images, and evaluate using the HPatches dataset [2].”
LF-Net: Learning Local Features from Images (2018, background): “…a performance evaluation for different pixel thresholds, to evaluate the precision of the detected features; results over the ‘hpatches’ dataset [4]; a study to benchmark performance under orientation changes; detailed results for the ablation experiments of Section 4.5; and computational cost…”
LoFTR: Detector-Free Local Feature Matching with Transformers (2021, methodology): “In the first experiment, we evaluate LoFTR on the widely adopted HPatches dataset [1] for homography estimation.”
Grounding Image Matching in 3D with MASt3R (2024, methodology): “Overall, keypoint-based approaches are predominant in many benchmarks [7,35,44,77], underscoring their enduring value in tasks requiring high precision and speed [19,77].”
BLINK: Multimodal Large Language Models Can See but Not Perceive (2024, background): “3 K images, where questions may contain multiple images that are curated from a wide range of datasets [8,10,19,29,31,35,41,46], encompassing indoor household scenes as well as outdoor urban or natural environments.”
TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement (2023, background): “This line of work is thus often restricted to rigid scenes [41, 80], and is benchmarked accordingly [1, 11, 31, 32, 56, 81].”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “The task of visual correspondence aims to find reliable correspondences under challenging degree of variations [3, 14, 15, 43, 60].”
Image Matching from Handcrafted to Deep Features: A Survey (2020, background): “…they merely cover a single part of image matching community, either focus on detectors (Huang et al. 2018; Lenc and Vedaldi 2014) or descriptors (Balntas et al. 2017; Schonberger et al. 2017) or specific matching tasks (Ferrante and Paragios 2017; Haskins et al. 2020; Yan et al. 2016b; Maiseli…”
Learning and aggregating deep local descriptors for instance-level recognition (2020, background): “Local descriptor dimensions are de-correlated by a linear whitening transformation; PCA-whitening improves the discriminability of both local [6] and global descriptors [36].”
Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters (2019, methodology): “HPatches [2] dataset is used for testing.”
TAP-Vid: A Benchmark for Tracking Any Point in a Video (2022, neutral): “This work is thus often restricted to rigid scenes [45, 87], and is benchmarked accordingly [3, 10, 36, 36, 37, 57, 88].”
Working hard to know your neighbor's margins: Local descriptor learning loss (2017, methodology): “We refer the reader to the HPatches paper [18] for a detailed protocol for each task.”
Emergent Correspondence from Image Diffusion (2023, methodology): “We evaluate DIFT for homography estimation using the HPatches benchmark [4].”
SuperGlue: Learning Feature Matching With Graph Neural Networks (2019, methodology): “HPatches: We assess the generalization ability of SuperGlue on real data with the HPatches [3] dataset, as done in previous works [18, 45].”