Wide-Area Image Geolocalization with Aerial Reference Imagery

By Scott Workman, Richard Souvenir, Nathan Jacobs. 2015

462 citations recorded by the source corpus.

We propose to use deep convolutional neural networks to address the problem of cross-view image geolocalization, in which the geolocation of a ground-level query image is estimated by matching to georeferenced aerial images. We use state-of-the-art feature representations for ground-level images and introduce a cross-view training approach for learning a joint semantic feature representation for aerial images. We also propose a network architecture that fuses features extracted from aerial images at multiple spatial scales. To support training these networks, we introduce a massive database that contains pairs of aerial and ground-level images from across the United States. Our methods significantly out-perform the state of the art on two benchmark datasets. We also show, qualitatively, that the proposed feature representations are discriminative at both local and continental spatial scales.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, background): “Cross-view geolocation recognition makes additional use of satellite or aerial imagery to determine query locations [37,38,39,40].”
Revisiting IM2GPS in the Deep Learning Era (2017, methodology): “In [18, 34, 37], images of the same scene from the ground viewpoint and overhead viewpoint are embedded in the same feature space through deep learning DML; the resulting system then does localization by image retrieval using reference database of aerial images.”
PlaNet - Photo Geolocation with Convolutional Neural Networks (2016, methodology): “Exceptions are Im2GPS [18, 17] and [32, 33, 49], which make additional use of satellite imagery.”
Lending Orientation to Neural Networks for Cross-View Geo-Localization (2019, methodology): “They further extended the work to cross-view training and showed improved performance [34].”
Predicting Ground-Level Scene Layout from Aerial Imagery (2016, methodology): “Recent work on cross-view image geolocalization [17, 18, 32, 33] has shown that convolutional neural networks are capable of extracting features from aerial imagery that can be matched to features extracted from ground imagery.”
University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization (2020, methodology): “trastive loss [16, 36, 44].”
Spatial-Aware Feature Aggregation for Image based Cross-View Geo-Localization (2019, background): “The work [17] ﬁne-tunes CNNs by minimizing the feature distances between aerial and ground-view images and obtains better localization performance.”
VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval (2020, background): “Similar to CVUSA, CVACT [11] also consists of aligned panoramas and aerial images with ori- entation information.”
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017, unknown): “Workman et al. (2015) introduce another massive cross-view dataset.”
Each Part Matters: Local Patterns Facilitate Cross-View Geo-Localization (2020, methodology): “The performance has surpassed the reported result of other competitive methods such as [21], [20], [54], [55], [7], [4], and the proposed method outperforms the best method, i.”
CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial Geo-Localization (2018, methodology): “With the recent success of deep learning in many computer vision tasks, most of the existing works on cross-view image matching [49, 50, 46, 53] adopt the convolutional neural network (CNN) to learn representations for matching between ground and aerial images.”
Geolocation Estimation of Photos Using a Hierarchical Model and Scene Classification (2018, background): “For these reasons, the majority of approaches simplifies photo geolocalization by restricting the problem to urban photos of, for example, well-known landmarks and cities [3,25,34,43,45,48] or natural areas like deserts or mountains [5,33,38].”
Localizing and Orienting Street Views Using Overhead Imagery (2016, methodology): “Some similar attempts to collect a dataset for cross-view images matching task are [20, 34], but neither are publicly available.”
Deep learning in remote sensing: a review (2017, background): “Returning to more traditional CNNs, the authors of [154] and [155] study the specificity of images to refer to a given city: they study how closely images of Charleston, South Carolina, resemble those of San Francisco, and the other way around, by using the fully connected layers of Places CNN [156]”
Cross-View Image Matching for Geo-Localization in Urban Environments (2017, methodology): “In the datasets presented in [25] and [22], a large portion of the images do not contain any building.”