IM2GPS: estimating geographic information from a single image

By James Hays, Alexei A. Efros. 2008

1,051 citations recorded by the source corpus.

Estimating geographic information from an image is an excellent, difficult high-level computer vision problem whose time has come. The emergence of vast amounts of geographically-calibrated image data is a great reason for computer vision to start looking globally - on the scale of the entire planet! In this paper, we propose a simple algorithm for estimating a distribution over geographic locations from a single image using a purely data-driven scene matching approach. For this task, we leverage a dataset of over 6 million GPS-tagged images from the Internet. We represent the estimated image location as a probability distribution over the Earthpsilas surface. We quantitatively evaluate our approach in several geolocation tasks and demonstrate encouraging performance (up to 30 times better than chance). We show that geolocation estimates can provide the basis for numerous other image understanding tasks such as population density estimation, land cover estimation or urban/rural classification.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, positive): “However it is a new benchmark so to compare with prior work on geolocalization we also report results on the IM2GPS test set from Hays & Efros (2008) in Table 16.”
Shortcut learning in deep neural networks (2020, background): “Examples include using nearest neighbours for scene completion and estimating geolocation [81, 82], object recognition with local features only [36], reasoning based on single cue words [59, 14] or answering questions about a movie without ever showing the movie to a model [83].”
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, methodology): “Compared to naïve average of the underlying classiﬁers denoted by Average[1-5] and Average[1-2], CPlaNet[1-5] and CPlaNet[1-2] have ∼ 16 % and ∼ 13 % of accuracy gains at street level, respectively, compared to their counterparts.”
Revisiting IM2GPS in the Deep Learning Era (2017, methodology): “Comparing to Im2GPS [9, 10], deep learning feature extraction is orders of magnitudes faster than computing many classical computer vision features.”
PlaNet - Photo Geolocation with Convolutional Neural Networks (2016, methodology): “One of the few approaches that, like ours, aims at geolocating arbitrary photos is Im2GPS [17, 18].”
YFCC100M (2015, background): “In the past, geotagged media has been used to address a variety of research endeavors, such as location estimation [9], event detection [15], ﬁnding canonical views of places [4], and visual reconstruction of the world [17].”
SUN database: Large-scale scene recognition from abbey to zoo (2010, methodology): “Sparse SIFT histograms: As in “Video Google” [27], we build SIFT features at Hessian-affine and MSER [19] interest points.”
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017, background): “PlaNet has been shown to successfully localize a large variety of photos and to out-perform Im2GPS [6, 7] that addresses the same task.”
Image-Based Recommendations on Styles and Substitutes (2015, background): “What distinguishes the approach we propose here, however, is the fact that it succeeds despite the indirect-ness of the connection between the dataset and the quantity we hope to model.”
Federated Visual Classification with Real-World Data Distribution (2020, background): “Many problems in visual classification involve data that vary around the globe [5,8].”
Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data (2021, background): “The pioneering work of Hays and Efros [9, 10] addresses geolocalization by image retrieval.”
A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation (2015, methodology): “We used the code framework by Hays and Efros [9]”
FlowNet: Learning Optical Flow with Convolutional Networks (2015, methodology): “We use the code framework by Hays and Efros [18] 4.”
Unsupervised Discovery of Mid-Level Discriminative Patches (2012, background): “formation at a pixel level to make a useful feature { it will re all the time. At the other extreme, one can use the whole image as a primitive which, while showing great promise in some applications [6,7], requires extraordinarily large amounts of training data, since one needs to represent all pos”
SUN Database: Exploring a Large Collection of Scene Categories (2014, methodology): “Line Hists: We find straight lines from Canny edges using the method described in Video Compass [23].”
Don't just listen, use your imagination: Leveraging visual common sense for non-visual tasks (2015, background): “Example tasks include reasoning about what people talk about in images [4], understanding the flow of time (when) [39], identifying where the image is taken [24, 28] and judging the intentions of people in images (why) [40].”
An empirical study of context in object detection (2009, methodology): “For semantic context, we use the keywords associated with matching scenes in the im2gps dataset [13] to predict object occurrence.”
Large-Scale Deep Learning on the YFCC100M Dataset (2015, methodology): “Others are interested in using the YFCC100M imagery and audio to geolocate where the photo or video was taken [14].”
End-to-End Learning of Deep Visual Representations for Image Retrieval (2016, methodology): “This has proved useful for annotations as diverse as image-level tags (Makadia et al, 2008), GPS coordinates (Hays and Efros, 2008), or prominent object location (RodriguezSerrano et al, 2015).”