Image Matching Across Wide Baselines: From Paper to Practice

By Yuhe Jin, Dmytro Mishkin, Anastasiia V. Mishchuk, Jiri Matas, Pascal Fua, Kwang Moo Yi, Eduard Trulls. International Journal of Computer Vision, 2020

469 citations recorded by the source corpus.

We introduce a comprehensive benchmark for local features and robust estimation algorithms, focusing on the downstream task—the accuracy of the reconstructed camera pose—as our primary metric. Our pipeline’s modular structure allows easy integration, configuration, and combination of different methods and heuristics. This is demonstrated by embedding dozens of popular algorithms and evaluating them, from seminal works to the cutting edge of machine learning research. We show that with proper settings, classical solutions may still outperform the perceived state of the art. Besides establishing the actual state of the art, the conducted experiments reveal unexpected properties of structure from motion pipelines that can help improve their performance, for both algorithmic and learned methods. Data and code are online (https://github.com/ubc-vision/image-matching-benchmark), providing an easy-to-use and flexible framework for the benchmarking of local features and robust estimation methods, both alongside and against top-performing methods. This work provides a basis for the Image Matching Challenge (https://image-matching-challenge.github.io).

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/021f956ab3bc13008e87d24f42feeced61eb5916

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Learned Initializations for Optimizing Coordinate-Based Neural Representations (2020, methodology): “For Phototourism landmark reconstruction, it can be optimized at test time to transfer the appearance of a single input image onto the whole landmark, which can then be rendered from novel camera views.”
COTR: Correspondence Transformer for Matching Across Images (2021, methodology): “One is to extract sets of sparse keypoints from both images and match them in order to minimize an alignment metric [33, 55, 28].”
Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians (2024, background): “Balancing anchors in each LOD level and adjusting training iteration activation are also crucial.”
DISK: Learning local features with policy gradient (2020, methodology): “We report the results in Table 1, along with baselines taken directly from the leaderboards, computed in [16].”
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections (2020, methodology): “For the former, we select three landmarks from the Phototourism dataset [13].”
Image Matching from Handcrafted to Deep Features: A Survey (2020, background): “Jin et al. (2020) introduced a benchmark for local features and robust estimation algorithms, focusing on the accuracy of the reconstructed camera pose as their practical evaluation.”
LightGlue: Local Feature Matching at Light Speed (2023, methodology): “[68] to avoid training on scenes included in the Image Matching Challenge [31].”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, unknown): “NeRF-W did not improve on NeRF in terms of rendering speed, but achieved much higher results in the crowded Phototourism dataset [121].”
Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly- Throughs (2021, background): “Although it does not explicitly target unbounded scenes, it achieves impressive results against outdoor sequences in the Phototourism [15] dataset.”
Grounding Image Matching in 3D with MASt3R (2024, background): “Overall, keypoint-based approaches are predominant in many benchmarks [7,35,44,77], underscoring their enduring value in tasks requiring high precision and speed [19,77].”