Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions

By Torsten Sattler, William P. Maddern, Carl Toft, Akihiko Torii, Lars Hammarstrand, Erik Stenborg, Daniel Safari, Masatoshi Okutomi, Marc Pollefeys, Josef Sivic and 2 more. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

770 citations recorded by the source corpus.

Visual localization enables autonomous vehicles to navigate in their surroundings and augmented reality applications to link virtual to real worlds. Practical visual localization approaches need to be robust to a wide variety of viewing condition, including day-night changes, as well as weather and seasonal variations, while providing highly accurate 6 degree-of-freedom (6DOF) camera pose estimates. In this paper, we introduce the first benchmark datasets specifically designed for analyzing the impact of such factors on visual localization. Using carefully created ground truth poses for query images taken under a wide variety of conditions, we evaluate the impact of various factors on 6DOF camera pose estimation accuracy through extensive experiments with state-of-the-art localization approaches. Based on our results, we draw conclusions about the difficulty of different conditions, showing that long-term localization is far from solved, and propose promising avenues for future work, including sequence-based localization approaches and the need for better local features. Our benchmark is available at visuallocalization.net.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5b9b5b7e20afff4e2caa08e0d22cefe87007bf63

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Back to the Feature: Learning Robust Camera Localization from Pixels to Pose (2021, methodology): “It is initialized by the weighted average [66] of the top-3 poses retrieved by NetVLAD for Aachen and top-1 for RobotCar and CMU.”
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression (2019, background): “At the same time, DenseVLAD has been shown to perform well even on challenging localization tasks [60, 61, 71].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “Similarly, the visual localization challenge and the corresponding benchmark [96] cover weather and season diversity (but no new multi-modal dataset is introduced).”
LightGlue: Local Feature Matching at Light Speed (2023, methodology): “SuperGlue is highly effective for visual localization in challenging conditions [59, 55, 58, 57] and generalizes well to other tasks like aerial matching [83], object pose estimation [69], and even fish re-identification [47].”
D2-Net: A Trainable CNN for Joint Description and Detection of Local Features (2019, methodology): “Our sparse feature approach even outperforms DenseSfM, even though the later is using sig-niﬁcantly more features (and thus time and memory).”
ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer (2022, background): “1 [54,3,55] datasets, to demonstrate performance on multi-view matching in indoor scenes and outdoor scenes, respectively.”
R2D2: Repeatable and Reliable Detector and Descriptor (2019, methodology): “We also use pairs extracted (with the help of SfM) from the Aachen Day-Night dataset [44, 46] which contains images from the old inner city of Aachen, Germany.”
The ApolloScape Open Dataset for Autonomous Driving and Its Application (2018, background): “Most recently, various datasets targeting at solving each individual visual task for robot navigation have been released such as 3D geometry estimation [32], [33], localization [9], [34], instance detection and segmentation [35], [36].”
LPD-Net: 3D Point Cloud Learning for Large-Scale Place Recognition and Environment Analysis (2018, methodology): “This comparison is conducted on the Robotcar Seasons dataset [16], and we generate the corresponding point clouds by using the original data from the Robotcar dataset.”
Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition (2021, methodology): “Traditional handcrafted local feature methods such as SIFT [44], SURF [6] and ORB [60], and more recent deep-learned local features like LIFT [84], DeLF [52], SuperPoint [18] and D2Net [20], have been extensively employed for VPR [52, 9, 17], visual SLAM [50] and 6-DoF localization [63, 77, 20, 61].”
Neighbourhood Consensus Networks (2018, background): “results open up the possibility for end-to-end learning of other challenging visual correspondence tasks, such as 3D category-level matching [18], or visual localization across day/night illumination [31]. Acknowledgements This work was partially supported by JSPS KAKENHI Grant Numbers 15H05313, 16K”
SuperGlue: Learning Feature Matching With Graph Neural Networks (2019, background): “ localization Visual localization: While two-view relative pose estimation is an important fundamental problem, advances in image matching can directly beneﬁt practical tasks like visual localization [50,48], which aims at estimating the absolute pose of a query image with respect to a 3D model. Mor”
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, methodology): “Contrary to these domain-specific methods, previous work also includes both methods designed for robustness to illumination changes, by employing domaininvariant representations [1, 25] or fusing information from complementary modalities and spectra [33], and datasets with adverse illumination [21, ”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “The task of visual correspondence aims to find reliable correspondences under challenging degree of variations [3, 14, 15, 43, 60].”