Tour the world: Building a web-scale landmark recognition engine

By Yantao Zheng, Ming Zhao, Yang Song, Hartwig Adam, Ulrich Buddemeier, Alessandro Bissacco, Fernando Brucher, Tat-seng Chua, Hartmut Neven. 2009

423 citations recorded by the source corpus.

Modeling and recognizing landmarks at world-scale is a useful yet challenging task. There exists no readily available list of worldwide landmarks. Obtaining reliable visual models for each landmark can also pose problems, and efficiency is another challenge for such a large scale system. This paper leverages the vast amount of multimedia data on the Web, the availability of an Internet image search engine, and advances in object recognition and clustering techniques, to address these issues. First, a comprehensive list of landmarks is mined from two sources: (1) ~20 million GPS-tagged photos and (2) online tour guide Web pages. Candidate images for each landmark are then obtained from photo sharing Websites or by querying an image search engine. Second, landmark visual models are built by pruning candidate images using efficient image matching and unsupervised clustering techniques. Finally, the landmarks and their visual models are validated by checking authorship of their member images. The resulting landmark recognition engine incorporates 5312 landmarks from 1259 cities in 144 countries. The experiments demonstrate that the engine can deliver satisfactory recognition performance with high efficiency.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, methodology): “The database serves as the index of an image retrieval sys-tem [28,29,30,31,32,33] or the training data of a landmark classiﬁer [34,35,36].”
PlaNet - Photo Geolocation with Convolutional Neural Networks (2016, background): “Most previous work has therefore focused on covering restricted subsets of the problem, like landmark build-1 ings [1, 37, 54], cities where street view imagery is available [7, 26, 52], or places where coverage of internet photos is dense enough to allow building a structure-from-motion…”
Photo2Trip: generating travel routes from geo-tagged photos for trip planning (2010, background): “[10] mined city landmarks from blogs by exploiting graphic models, while [8], [13], [11] and [5] attempted to visualize, recognize, describe and summarize a scene or a landmark by leveraging geo-tagged photos.”
Graph-Based Discriminative Learning for Location Recognition (2013, background): “, the Eiffel Tower can be represented with a collection of photos [33]).”
Exhaustive Geometric Veriﬁcation for Image Retrieval Accepted for ECCV 2012 (methodology): “It then describes each feature using a descriptor similar to the one presented in [21], where we have removed rotational invariance.”
Retrieving landmark and non-landmark images from community photo collections (2010, methodology): “On the other hand, a second layer of visual clustering follows in other approaches, using different algorithms including k-means ([15]) and agglomerative clustering ([27], [8], [36]).”
Image webs: Computing and exploiting connectivity in image collections (2010, methodology): “As we briefly survey in Section 2, considerable research has already been devoted to exploiting match links to detect images of popular landmarks [25] and to annotate them [11], or for 3-D reconstruction of landmarks [23].”
Clues from the beaten path: Location estimation with bursty sequences of tourist photos (2011, background): “Recent work shows that with databases containing millions of geo-tagged images, one can employ simpler global image descriptors (e.g., Gist, bag-of-words histograms) and still retrieve relevant scenes for a novel photo [6, 9, 7, 22 ].”
Landmark classification in large-scale image collections (2009, background): “, [2, 14, 21]), however such approaches generally make use of image features and machine learning techniques, and thus the resulting datasets are not independent of the kinds of features and methods that one wants to test.”
Location Recognition Using Prioritized Feature Matching (2010, background): “Similarly, [19] builds a landmark recognition engine by selecting iconic images using a graph-based algorithm.”
Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images (2017, neutral): “Privacy and Computer Vision Several works explore detecting individual privacy attributes such as license plates [53, 49, 6], age estimation from facial photographs [4], social relationships [45], face detection [40, 44], landmark detection [51] and occupation recognition [39].”
The Placing Task: A Large-Scale Geo-Estimation Challenge for Social-Media Videos and Images (2014, methodology): “A web-scale landmark-recognition engine called “Tour the World” was built by using 20 million GPS-tagged photos of landmarks cross-referenced with online tour guide web pages [18].”
Leveraging Structure from Motion to Learn Discriminative Codebooks for Scalable Landmark Classification (2013, methodology): “Unlike [16, 37, 12], we did not use geo-tags, since we found that geo-tags are not always representative of the image contents, which was also observed by [12].”
Visual landmark recognition from Internet photo collections: A large-scale evaluation (2014, methodology): “of oine query expansion. However, like query expansion, this method is also prone to drift (Fig.16, top row), which can cause confusion between nearby objects. While the often used Best Match method [4,6,13,2] avoids drift better than some other methods, we found that by using a method that explicit”
Worldwide Pose Estimation Using 3D Point Clouds (2012, neutral): “, the “landmark recognition/classification” problem [12, 13].”
Social Network Data Analytics (2011, background): “Weiss and Jacobson examined work groups within a government agency[123].”