Leveraging Structure from Motion to Learn Discriminative Codebooks for Scalable Landmark Classification

By Alessandro Bergamo, Sudipta N. Sinha, Lorenzo Torresani. 2013

54 citations recorded by the source corpus.

In this paper we propose a new technique for learning a discriminative codebook for local feature descriptors, specifically designed for scalable landmark classification. The key contribution lies in exploiting the knowledge of correspondences within sets of feature descriptors during code-book learning. Feature correspondences are obtained using structure from motion (SfM) computation on Internet photo collections which serve as the training data. Our codebook is defined by a random forest that is trained to map corresponding feature descriptors into identical codes. Unlike prior forest-based codebook learning methods, we utilize fine-grained descriptor labels and address the challenge of training a forest with an extremely large number of labels. Our codebook is used with various existing feature encoding schemes and also a variant we propose for importance-weighted aggregation of local features. We evaluate our approach on a public dataset of 25 landmarks and our new dataset of 620 landmarks (614K images). Our approach significantly outperforms the state of the art in landmark classification. Furthermore, our method is memory efficient and scalable.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, methodology): “The database serves as the index of an image retrieval system [28,29,30,31,32,33] or the training data of a landmark classifier [34,35,36].”
PlaNet - Photo Geolocation with Convolutional Neural Networks (2016, methodology): “Instead of us-ing image retrieval, [4, 29] use SVMs trained on BoVW of landmark clusters to decide which landmark is shown in a query image.”
Multi-scale Orderless Pooling of Deep Convolutional Activation Features (2014, methodology): “For this, we adopt Vectors of Locally Aggregated Descriptors (VLAD) [16,25], which are a simplified version of Fisher Vectors (FV) [15].”
AutoCaption: Automatic caption generation for personal photos (2014, methodology): “We currently perform face detection [26, 20] and recognition [4], proxemics detection [24], scene classification [23], landmarks detection [2], process the date and time meta-data, and analyze the GPS information.”
Visual landmark recognition from Internet photo collections: A large-scale evaluation (2014, methodology): “[30] use a similar approach, but perform classification using 1vs-all SVMs.”
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization (2015, methodology): “Previous research such as [27, 14, 9, 3] has also used SIFT-like point based features to match and localize from landmarks.”