Image Retrieval for Image-Based Localization Revisited

By Torsten Sattler, Tobias Weyand, Bastian Leibe, Leif Kobbelt. 2012

314 citations recorded by the source corpus.

To reliably determine the camera pose of an image relative to a 3D point cloud of a scene, correspondences between 2D features and 3D points are needed. Recent work has demonstrated that directly matching the features against the points outperforms methods that take an intermediate image retrieval step in terms of the number of images that can be localized successfully. Yet, direct matching is inherently less scalable than retrievalbased approaches. In this paper, we therefore analyze the algorithmic factors that cause the performance gap and identify false positive votes as the main source of the gap. Based on a detailed experimental evaluation, we show that retrieval methods using a selective voting scheme are able to outperform state-of-the-art direct matching methods. We explore how both selective voting and correspondence computation can be accelerated by using a Hamming embedding of feature descriptors. Furthermore, we introduce a new dataset with challenging query images for the evaluation of image-based localization.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, methodology): “Instead of directly matching against a 3D model, [24, 23] first perform image retrieval to obtain coarse locations and then estimate poses using the retrieved images.”
PlaNet - Photo Geolocation with Convolutional Neural Networks (2016, methodology): “Because matching the query descriptors against the 3D model descriptors is expensive, some approaches combine this technique with efﬁ-cient image retrieval based on inverted indices [5, 21, 42].”
NetVLAD: CNN Architecture for Weakly Supervised Place Recognition (2015, background): “shelf CNN descriptors on two challenging place recognition benchmarks. 1. Introduction Visual place recognition has received a signiﬁcant amount of attention in the past years both in computer vision [4,9,10,23,33,58,59,60,61,75,76] and robotics communities [15,16,40,42,70] motivated by, e.g., appli”
Learned Contextual Feature Reweighting for Image Geo-Localization (2017, methodology): “Image geo-localization has been cast as an image retrieval task [2, 10, 19, 28, 66], a 2D to 3D registration task [17, 21, 32, 47, 46], and a per-location classification task [16, 62].”
D2-Net: A Trainable CNN for Joint Description and Detection of Local Features (2019, methodology): “Section D shows qualitative examples for the matches found with our approach on the InLoc [59] and Aachen Day-Night [46,48] datasets.”
SuperGlue: Learning Feature Matching With Graph Neural Networks (2019, methodology): “ higher scene diversity and more challenging conditions, such as larger viewpoint and illumination changes, than phototourism datasets of popular landmarks. Evaluation: The Aachen Day-Night benchmark [51,50] evaluates local feature matching for day-night localization. We extract up to 4096 keypoints”
ASLFeat: Learning Local Features of Accurate Shape and Localization (2020, methodology): “Designing powerful local features is an essential basis for a broad range of computer vision tasks [33, 47, 48, 32, 44, 17, 44].”
Back to the Feature: Learning Robust Camera Localization from Pixels to Pose (2021, methodology): “Similarly, we show successful and failed examples on the Aachen Day-Night dataset in Figures 12 and 13, respectively.”
From Coarse to Fine: Robust Hierarchical Localization at Large Scale (2018, background): “The Aachen Day-Night dataset [45] contains 4,328 day-time database images from a European old town, and 824 and 98 queries taken in day and night conditions respectively.”
Efficient Global 2D-3D Matching for Camera Localization in a Large-Scale 3D Map (2017, methodology): “, L1 norm distances between Bag-of-Words vectors [19], L2 norm distances between compact representations/vectors [6, 7]), and the position of the best database image is deemed as that of the query image or the top N images are fused to get the position of the query image [46, 47, 50].”
Visual Place Recognition with Repetitive Structures (2015, background): “This natural soft-assignment is more precise and less ambiguous than the standard soft-assignment to multiple nearest visual words [27] as will be demonstrated in the next section.”
R2D2: Repeatable and Reliable Detector and Descriptor (2019, methodology): “We also use pairs extracted (with the help of SfM) from the Aachen Day-Night dataset [44, 46] which contains images from the old inner city of Aachen, Germany.”
Graph-Based Discriminative Learning for Location Recognition (2013, methodology): “We evaluate our algorithm on the Dubrovnik and Rome datasets [17] and the Aachen dataset [26]; these are summarized in Table 1, along with statistics over the neighborhoods we compute.”
Visual landmark recognition from Internet photo collections: A large-scale evaluation (2014, background): “uate object recognition, i.e. recognizing the object(s) in a query image, and not image retrieval, i.e. retrieving images similar to a query from a database. Image-based localization datasets (Aachen [19], Vienna [20], Dubrovnik and Rome [21]) evaluate how accurately the camera pose of the query ima”
ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer (2022, background): “Image matching lays the foundation for various geometric computer vision tasks, including Structure from Motion (SfM) [1,2], visual localization [3], and Simultaneous Localization And Mapping (SLAM) [4,5].”
Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions (2018, methodology): “Our Aachen Day-Night dataset is based on the Aachen localization dataset from [58].”
Predicting Good Features for Image Geo-Localization Using Per-Bundle VLAD (2015, methodology): “The other is to estimate the full camera pose of the query image using a 3D structure-from-motion model constructed from reference images [13, 17, 25, 31], which is limited to places with a dense distribution of reference images.”
24/7 Place Recognition by View Synthesis (2018, unknown): “RECENT years have seen a tremendous progress [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16] in the large-scale visual place recognition problem [14], [17].”
Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization (2015, methodology): “the query keyframe and the top-ranked retrieved images is performed to reduce the fraction of outliers [20, 43].”