TILDE: A Temporally Invariant Learned DEtector

By Yannick Verdie, Kwang Moo Yi, Pascal Fua, Vincent Lepetit. Computer Vision and Pattern Recognition, 2014

349 citations recorded by the source corpus.

We introduce a learning-based approach to detect repeatable keypoints under drastic imaging changes of weather and lighting conditions to which state-of-the-art keypoint detectors are surprisingly sensitive. We first identify good keypoint candidates in multiple training images taken from the same viewpoint. We then train a regressor to predict a score map whose maxima are those points so that they can be found by simple non-maximum suppression. As there are no standard datasets to test the influence of these kinds of changes, we created our own, which we will make publicly available. We will show that our method significantly outperforms the state-of-the-art methods in such challenging conditions, while still achieving state-of-the-art performance on untrained standard datasets.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f7eceb72c0e1d432b0f96b985a5305a669b55119

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Partially Does It: Towards Scene-Level FG-SBIR with Partial Input (2022, background): “Region-wise correspondence could be learned in a supervised [70,78,84], self-supervised [19, 40, 83], or unsupervised manner [3, 28, 35,52,61].”
3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration (2018, background): “TILDE [31] learns to detect keypoints that can be reliably matched over diﬀerent lighting conditions.”
Image Matching Across Wide Baselines: From Paper to Practice (2020, methodology): “From left to right: (a) VGG Oxford [59], (b) HPatches [11], (c) Edge Foci [121], (d) Webcam [106], (e) AMOS [47], (f) Kitti [41], (g) Strecha [93], (h) SILDa [10], (i) Aachen [83], (j) Ours.”
Grounding Image Matching in 3D with MASt3R (2024, methodology): “Departing from the former handcrafted methods like SIFT [52,71], modern approaches have been shifting towards learning-based data-driven schemes for detecting keypoints [8,60,97,117], describing them [7,33, 37,88] or both at the same time [10,21,53,54,70,98].”
Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking (2018, background): “Arc de Triomphe has three day and two night queries, while day-night matching is considered a challenging problem [49, 35].”
LIFT: Learned Invariant Feature Transform (2016, methodology): “In contrast to these approaches, that focus on better engineering, it was recently shown that a detector could be learned [6] and deliver significantly better performance than the state-of-the-art.”
Large-Scale Image Retrieval with Attentive Deep Local Features (2016, methodology): “ve re-evaluated and proposed different feature aggregation methods using such deep local features [3,21]. CNNs have also been used to detect, represent and compare local image features. Verdie et al. [37] learned a regressor for repeatable keypoint detection. Yi et al. [41] proposed a generic CNN-ba”
Learning to Detect Good 3D Keypoints (2017, methodology): “A different approach is proposed in Verdie et al. (2015),where themost repeatableDoGkeypoints across a set of training images are used to define the positive samples to train a saliency function able to highlight the same image points under drastic illumination changes caused by weather, season and…”
LF-Net: Learning Local Features from Images (2018, methodology): “However, many other approaches target some of their individual components, be it feature point extraction [31, 44], orientation estimation [50], or descriptor generation [36, 40, 41].”
SuperPoint: Self-Supervised Interest Point Detection and Description (2017, methodology): “Interest Points? Descriptors? Full Image Input? Single Network? Real Time? SuperPoint (ours) ✓ ✓ ✓ ✓ ✓ LIFT [32] ✓ ✓ UCN [3] ✓ ✓ ✓ TILDE [29] ✓ ✓ DeepDesc [6] ✓ ✓ SIFT ✓ ✓ ORB ✓ ✓ ✓”
Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters (2019, methodology): “A number of related works have focused their objective function to address this issue, although they were based either on local patches [9, 10] or global map regression loss [13, 14, 11].”
Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “Inspired by the handcrafted feature detectors, a general solution for CNN-based detection is to construct response maps to search the interest points in a supervised (Yi et al. 2016; Verdie et al. 2015; Zhang et al. 2017b), self-supervised (Zhang and Rusinkiewicz 2018; DeTone et al.”
A review of multimodal image matching: Methods and applications (2021, methodology): “[182] trained multiple piecewise linear regression models from aligned image pairs by using handcrafted DoG for training set collection and then training general regressors to generate a feature score map.”