Large-Scale Image Retrieval with Attentive Deep Local Features

By Hyeonwoo Noh, Andre F. de Araújo, Jack Sim, Tobias Weyand, Bohyung Han. 2016

860 citations recorded by the source corpus.

We propose an attentive local feature descriptor suitable for large-scale image retrieval, referred to as DELE (DEep Local Feature). The new feature is based on convolutional neural networks, which are trained only with image-level annotations on a landmark image dataset. To identify semantically useful local features for image retrieval, we also propose an attention mechanism for key point selection, which shares most network layers with the descriptor. This frame-work can be used for image retrieval as a drop-in replacement for other keypoint detectors and descriptors, enabling more accurate feature matching and geometric verification. Our system produces reliable confidence scores to reject false positives–in particular, it is robust against queries that have no correct match in the database. To evaluate the proposed descriptor, we introduce a new large-scale dataset, referred to as Google-Landmarks dataset, which involves challenges in both database and query such as background clutter, partial occlusion, multiple landmarks, objects in variable scales, etc. We show that DELE outperforms the state-of-the-art global and local descriptors in the large-scale setting by significant margins.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, background): “DELF [17] is a deeply learned local image feature detector and descriptor with attention for image retrieval.”
A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020, background): “• Google Landmarks [33] ( LANDMARKS below) is built for landmark recognition, and we selected 1000 classes having the largest number of associated images.”
Unifying Deep Local and Global Features for Image Search (2020, methodology): “Local features [28,7,63,39,34], on the other hand, comprise descriptors and geometry information about specific image regions; they are especially useful to match images depicting rigid objects.”
Learning With Average Precision: Training Image Retrieval With a Listwise Loss (2019, neutral): “[41], which is based on local descriptors.”
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation (2021, background): “Part-level features have been more widely used in the domain of image retrieval with both classical [34, 5] and deep learning based [2, 37, 42, 45, 7] methods.”
Fine-Tuning CNN Image Retrieval with No Human Annotation (2017, methodology): “[47] to learn global image descriptors using a saliency mask.”
PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors (2018, background): “Already in 2D, learned descriptors significantly outperform their engineered counterparts [49, 28].”
DISK: Learning local features with policy gradient (2020, methodology): “Our method is #1 on the viewpoint scenes, followed by R2D2, and #2 on the illumination scenes, trailing DELF.”
Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining (2021, methodology): “Intra-modal retrieval [32, 1] has been extensively studied in the keyword-based web document retrieval [11], content-based image retrieval [29], and product recommendation [19, 20].”
Initial Classifier Weights Replay for Memoryless Class Incremental Learning (2020, neutral): “• Google Landmarks [21] ( Landmarks below) is built for landmark recognition, and we selected 1000 classes having the largest number of associated images.”
Learning and aggregating deep local descriptors for instance-level recognition (2020, methodology): “posed method has connections to dierent approaches in the literature which are discussed in this section. The work closest to ours, in terms of training of the local detector and descriptor, is DELF [29]. Following our notation, DELF generates local feature descriptors Z ^h^ow^ (U), where h^(:) is i”
From Coarse to Fine: Robust Hierarchical Localization at Large Scale (2018, methodology): “We thus train on 185k images from the Google Landmarks dataset [36], containing a wide variety of day-time urban scenes, and 37k images from the night and dawn sequences of the Berkeley Deep Drive dataset [58], composed of road scenes with motion blur.”