Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters

By Axel Barroso Laguna, Edgar Riba, Daniel Ponsa, Krystian Mikolajczyk. IEEE International Conference on Computer Vision, 2019

327 citations recorded by the source corpus.

We introduce a novel approach for keypoint detection task that combines handcrafted and learned CNN filters within a shallow multi-scale architecture. Handcrafted filters provide anchor structures for learned filters, which localize, score and rank repeatable features. Scale-space representation is used within the network to extract keypoints at different levels. We design a loss function to detect robust features that exist across a range of scales and to maximize the repeatability score. Our Key.Net model is trained on data synthetically created from ImageNet and evaluated on HPatches benchmark. Results show that our approach outperforms state-of-the-art detectors in terms of repeatability, matching performance and complexity.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/676043564bb46dcbe1c94c7ea668cf596bb9b95a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

R2D2: Repeatable and Reliable Detector and Descriptor (2019, background): “In [22], handcrafted and learned filters are combined to detect repeatable keypoints.”
Image Matching Across Wide Baselines: From Paper to Practice (2020, methodology): “There have been multiple attempts to learn keypoint detectors separately from the descriptor, including TILDE [95], TCDet [103], Quad-Net [78], and KeyNet [13].”
DISK: Learning local features with policy gradient (2020, background): “Other learning-based methods focus on keypoints [42, 35, 18] or orientations [47], or merge the two notions entirely [8].”
Grounding Image Matching in 3D with MASt3R (2024, methodology): “Departing from the former handcrafted methods like SIFT [52,71], modern approaches have been shifting towards learning-based data-driven schemes for detecting keypoints [8,60,97,117], describing them [7,33, 37,88] or both at the same time [10,21,53,54,70,98].”
ASLFeat: Learning Local Features of Accurate Shape and Localization (2020, background): “Net [14] learn grid-wise offsets to localize keypoints.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “…but the performance could be largely restricted by the method of anchor construction, because the anchor itself is intrinsically difficult to reasonably define and may pre- vent the network from proposing new keypoints in case no anchor exists in the proximity (Barroso-Laguna et al. 2019).”
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input (2022, background): “Region-wise correspondence could be learned in a supervised [69,77,83], self-supervised [18, 39, 82], or unsupervised manner [3, 27, 34,51,60].”
DUSt3R: Geometric 3D Vision Made Easy (2023, unknown): “The traditional pipeline starts from pixel correspondences obtained from keypoint matching [4, 5, 43, 63, 99] between multiple images to determine geometric relationships, followed by bundle adjustment to optimize 3D coordinates and camera parameters jointly.”
Kornia: an Open Source Differentiable Computer Vision Library for PyTorch (2019, background): “—kornia.features— provides a high level API to perform detections in scale-space, where classical hard non-maxima suppression is replaced with its soft version similar to the recently proposed Multiscale Index Proposal layer (M-SIP) [30].”
Unifying Deep Local and Global Features for Image Search (2020, methodology): “matching, often employing RANSAC [15]. This has been widely used [41, 42, 3], as it produces reliable and interpretable scores. Recently, several deep learning-based local features have been proposed [58, 37, 32, 40, 33, 46, 14, 6]. The one most related to our work is DELF [37]; our proposed uniﬁed ”