Revisiting IM2GPS in the Deep Learning Era

By Nam N. Vo, Nathan Jacobs, James Hays. 2017

182 citations recorded by the source corpus.

Image geolocalization, inferring the geographic location of an image, is a challenging computer vision problem with many potential applications. The recent state-of-the-art approach to this problem is a deep image classification approach in which the world is spatially divided into cells and a deep network is trained to predict the correct cell for a given image. We propose to combine this approach with the original Im2GPS approach in which a query image is matched against a database of geotagged images and the location is inferred from the retrieved set. We estimate the geographic location of a query image by applying kernel density estimation to the locations of its nearest neighbors in the reference database. Interestingly, we find that the best features for our retrieval task are derived from networks trained with classification loss even though we do not use a classification approach at test time. Training with classification loss outperforms several deep feature learning methods (e.g. Siamese networks with contrastive of triplet loss) more typical for retrieval applications. Our simple approach achieves state-of-the-art geolocalization accuracy while also requiring significantly less training data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, methodology): “The latter treats the task as a classification problem by dividing the map into multiple discrete classes [3, 4].”
Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data (2021, methodology): “[32] replace handcrafted retrieval features with ones computed by a deep neural network.”
Geography-Aware Self-Supervised Learning (2020, background): “Other studies [38, 10, 11, 36] use geo-tagged training datasets to learn how to predict the geo-location of previously unseen images at test time.”
Learning and aggregating deep local descriptors for instance-level recognition (2020, methodology): “etter performance than cross entropy loss, in a classication manner, as done by Noh et al. [29] (4 vs 5). This is a conrmation of results that appear in the literature of instance-level recognition [14,46]. In Figure 5, we present the evolution of the model during training. We evaluate the performan”
Talk the Walk: Navigating New York City through Grounded Dialogue (2018, background): “Also the task of localization (without multi-agent communication) has recently been studied (Chaplot et al., 2018a; Vo et al., 2017).”
CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial Geo-Localization (2018, background): “Traditional image-based geolocalization is normally done in the context where both the query and geo-tagged reference images in the database are taken from the ground view [16, 52, 33, 45].”
ClusterFit: Improving Generalization of Visual Representations (2019, background): “Proxy tasks include hashtags predictions [11, 20, 25, 38, 56], GPS [27, 58], search queries prediction [51], and word or n-grams predictions [31, 36].”
Geolocation Estimation of Photos Using a Hierarchical Model and Scene Classification (2018, methodology): “Moreover, geographical partitioning approaches [39,42] entail a trade-off problem.”