Visual landmark recognition from Internet photo collections: A large-scale evaluation

By Tobias Weyand, Bastian Leibe. 2014

38 citations recorded by the source corpus.

Abstract The task of a visual landmark recognition system is to identify photographed buildings or objects in query photos and to provide the user with relevant information on them. With their increasing coverage of the world’s landmark buildings and objects, Internet photo collections are now being used as a source for building such systems in a fully automatic fashion. This process typically consists of three steps: clustering large amounts of images by the objects they depict; determining object names from user-provided tags; and building a robust, compact, and efficient recognition index. To this date, however, there is little empirical information on how well current approaches for those steps perform in a large-scale open-set mining and recognition task. Furthermore, there is little empirical information on how recognition performance varies for different types of landmark objects and where there is still potential for improvement. With this paper, we intend to fill these gaps. Using a dataset of 500 k images from Paris, we analyze each component of the landmark recognition pipeline in order to answer the following questions: How many and what kinds of objects can be discovered automatically? How can we best use the resulting image clusters to recognize the object in a query? How can the object be efficiently represented in memory for recognition? How reliably can semantic information be extracted? And finally: What are the limiting factors in the resulting pipeline from query to semantics? We evaluate how different choices of methods and parameters for the individual pipeline steps affect overall system performance and examine their effects for different query categories such as buildings, paintings or sculptures.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, methodology): “The database serves as the index of an image retrieval sys-tem [28,29,30,31,32,33] or the training data of a landmark classiﬁer [34,35,36].”
Google Landmarks Dataset v2 – A Large-Scale Benchmark for Instance-Level Recognition and Retrieval (2020, background): “7M StreetView City Y Landmarks-PointCloud [36] 2012 1k 10k 205k Flickr label + SfM Worldwide Y 24/7 Tokyo [56] 2015 125 315 1k Smartphone + Manual City Y Paris500k [61] 2015 13k 3k 501k Manual City N Landmark URLs [7, 22] 2016 586 140k - Text query + Feature matching Worldwide N Flickr-SfM [45] 2016”
Egocentric Visitors Localization in Cultural Sites (2019, background): “For instance, previous works addressed visual landmark recognition with smartphones [1, 15, 28].”