From Coarse to Fine: Robust Hierarchical Localization at Large Scale

By Paul-Edouard Sarlin, César Cadena, Roland Yves Siegwart, Marcin Dymczyk. Computer Vision and Pattern Recognition, 2018

1,175 citations recorded by the source corpus.

Robust and accurate visual localization is a fundamental capability for numerous applications, such as autonomous driving, mobile robotics, or augmented reality. It remains, however, a challenging task, particularly for large-scale environments and in presence of significant appearance changes. State-of-the-art methods not only struggle with such scenarios, but are often too resource intensive for certain real-time applications. In this paper we propose HF-Net, a hierarchical localization approach based on a monolithic CNN that simultaneously predicts local features and global descriptors for accurate 6-DoF localization. We exploit the coarse-to-fine localization paradigm: we first perform a global retrieval to obtain location hypotheses and only later match local features within those candidate places. This hierarchical approach incurs significant runtime savings and makes our system suitable for real-time operation. By leveraging learned descriptors, our method achieves remarkable localization robustness across large variations of appearance and sets a new state-of-the-art on two challenging benchmarks for large-scale localization.

View Paper: https://www.semanticscholar.org/paper/97ef575d7049ba0f1aeba4453a6dcd5475567630

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Unifying Deep Local and Global Features for Image Search (2020, background): “Such a hybrid approach gained popularity in visual localization [55,50] and instance-level recognition problems [43,62].”
RoMa: Robust Dense Feature Matching (2023, unknown): “Following DKM, we use a canonical learning rate (for batchsize = 8 ) of 10 − 4 for the decoder, and 5 · 10 − 6 for the encoder(s).”
ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer (2022, methodology): “For both datasets, we use pretrained HLoc [57] to retrieve candidate pairs, and recover camera poses with the model trained on MegaDepth dataset following SuperGlue [30] and LoFTR [13].”
Kornia: an Open Source Differentiable Computer Vision Library for PyTorch (2019, background): “Local feature detectors and descriptors Local features are the workhorses of 3d reconstruction [49, 45], visual localization [48] and image retrieval [52].”
PGSR: Planar-Based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (2024, unknown): “In response, several approaches aim to enhance reconstruction completeness and accuracy by integrating deep neural networks into the matching process [41], [42].”
LightGlue: Local Feature Matching at Light Speed (2023, methodology): “We use the default recontruction pipeline of hloc [55] and retrieve 50 pairs per image using NetVLAD [1].”
Grounding Image Matching in 3D with MASt3R (2024, background): “Camera Pose estimation techniques vary widely, but the most successful strategies, for speed, accuracy and robustness trade-off, are fundamentally based on pixel matching [73, 75, 105].”
LPD-Net: 3D Point Cloud Learning for Large-Scale Place Recognition and Environment Analysis (2018, methodology): “Comparisons with vision-based methods (Ave recall @1 with different GPS location bounds: 3m/5m/10m/15m). dawn dusk overcast summer overcast winter night-rain sun night Our LPD-Net 65.1/79.7/86.5/88.4 64.7/79.9/87.3/89.8 63.5/79.7/85.3/86.8 45.6/73.8/79.2/81.0 20.1/32.8/40.6/44.6 74.1/82.3/87.8/89.4 ”
DUSt3R: Geometric 3D Vision Made Easy (2023, unknown): “Our method obtains comparable accuracy compared to existing approaches, being feature-matching ones [101, 103] or end-to-end learning-based methods [11, 55, 102, 125, 152], even managing to out-perform strong baselines like HLoc [101] in some cases.”