PlaNet - Photo Geolocation with Convolutional Neural Networks

By Tobias Weyand, Ilya Kostrikov, James Philbin. 2016

494 citations recorded by the source corpus.

Is it possible to determine the location of a photo from just its pixels? While the general problem seems exceptionally difficult, photos often contain cues such as landmarks, weather patterns, vegetation, road markings, or architectural details, which in combination allow to infer where the photo was taken. Previously, this problem has been approached using image retrieval methods. In contrast, we pose the problem as one of classification by subdividing the surface of the earth into thousands of multi-scale geographic cells, and train a deep network using millions of geotagged images. We show that the resulting model, called PlaNet, outperforms previous approaches and even attains superhuman accuracy in some cases. Moreover, we extend our model to photo albums by combining it with a long short-term memory (LSTM) architecture. By learning to exploit temporal coherence to geolocate uncertain photos, this model achieves a 50 % performance improvement over the single-image model.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, methodology): “While [44] looked at geo-localization, [23] utilized the YFCC-100M dataset [40] for representation learning.”
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, methodology): “The latter treats the task as a classification problem by dividing the map into multiple discrete classes [3, 4].”
Revisiting IM2GPS in the Deep Learning Era (2017, methodology): “To predict the location as precisely as possible, one would prefer a fine-grained partitioning (for example [36]’s partitioning has 26,263 cells).”
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017, methodology): “hin a certain distance from the ground truth. The numbers for the original PlaNet model are based on an updated version that has an improved architecture and training dataset. Scale Im2GPS [7] PlaNet [35] PlaNet MobileNet Continent (2500 km) 51.9% 77.6% 79.3% Country (750 km) 35.4% 64.0% 60.3% Regio”
Learning Transferable Architectures for Scalable Image Recognition (2017, methodology): “architectures [60, 29, 69]. The ImageNet results are particularly important because many state-of-theart computer vision problems (e.g., object detection [28], face detection [50], image localization [63]) derive image features or architectures from ImageNet classiﬁcation models. For instance, we ﬁn”
Large-Scale Evolution of Image Classifiers (2017, background): “Neural networks can successfully perform difﬁcult tasks where large amounts of training data are available (He et al., 2015; Weyand et al., 2016; Silver et al., 2016).”
Defeating Image Obfuscation with Deep Learning (2016, background): “Trained machine learning models now outperform humans on tasks such as object recognition [15, 20] and determining the geographic location of an image [49].”
Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data (2021, methodology): “Similar to the supervised learning results of PlaNet [33], most of the predictions were far from the actual location of the image because of the inherent difficulty of visual geolocalization.”
Cross-View Image Matching for Geo-Localization in Urban Environments (2017, background): “Recently, Weyand [24] introduced PlaNet, a deep learning model that integrates several cues from images, for photo geo-localization and demonstrated superior performance over IM2GPS [8].”
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression (2019, background): “Image retrieval is typically used for place recognition [3, 5, 17, 58, 71, 72, 77, 80], i.”
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision (2022, background): “Unlike previous works [108, 126, 127, 138], we neither make prior assumptions on the task nor simplify the problem by restricting the task to images from famous landmarks and cities, natural areas like deserts or mountains.”
Functional Map of the World (2017, background): “Perhaps the most similar work to ours in terms of temporal classification is PlaNet [26].”
Adversaries or allies? Privacy and deep learning in big data era (2018, unknown): “i Pr(classp = classi|o) log Pr(classp = classi|o), (2)”
Learning to Navigate in Cities Without a Map (2018, background): “For instance, convolutional neural networks (CNNs) achieve competitive scores on the geolocation task [43] and CNN+LSTM architectures improve on this [16, 32].”
Privacy Preserving Image-Based Localization (2019, methodology): “Traditional localization methods based on image retrieval [34,67] and based on learning [12, 35, 81, 82] have the advantage of not requiring the explicit storage of 3D maps.”