To Aggregate or Not to aggregate: Selective Match Kernels for Image Search

By Giorgos Tolias, Yannis Avrithis, Herv'e J'egou. 2013

232 citations recorded by the source corpus.

This paper considers a family of metrics to compare images based on their local descriptors. It encompasses the VLAD descriptor and matching techniques such as Hamming Embedding. Making the bridge between these approaches leads us to propose a match kernel that takes the best of existing techniques by combining an aggregation procedure with a selective match kernel. Finally, the representation underpinning this kernel is approximated, providing a large scale image search both precise and scalable, as shown by our experiments on several benchmarks.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

Citations in the corpus

CNN Features Off-the-Shelf: An Astounding Baseline for Recognition (2014, methodology): “Adopting the proposed pipeline of [18] and followed by others [16, 42] we process the extracted 4096 dim features in the following way: L2 normalize → PCA dimensionality reduction → whitening → L2 renormalization.”
NetVLAD: CNN Architecture for Weakly Supervised Place Recognition (2015, background): “While there have been many improvements in designing better image retrieval [3, 4, 12, 13, 18, 25, 26, 27, 29, 32, 48, 51, 52, 53, 54, 70, 77, 78, 81] and place recognition [5, 10, 11, 16, 17, 24, 35, 44, 46, 62, 63, 64, 74, 79, 80] systems, not many works have performed learning for these tasks.”
Multi-scale Orderless Pooling of Deep Convolutional Activation Features (2014, background): “[41] report 88% but their representation would take more than 4 million dimensions per image if expanded into an explicit feature vector, and is not scalable to large datasets.”
From generic to specific deep representations for visual recognition (2014, neutral): “non-ConvNet [35] [22] [40] [26] [38] [45] [25] [12] [18] [24] [42] [30] [37] [46] [37] [37] [4] 71.”
SIFT Meets CNN: A Decade Survey of Instance Retrieval (2016, background): “1), flat k-means can be adopted for codebook generation [85], [20].”
Web-Scale Image Clustering Revisited (2015, background): “Most aforementioned approaches use local features and descriptors, that hold the state-of-the-art in image retrieval but incur significant space and time cost [34].”
Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking (2018, methodology): “A typical pipeline consists of invariant local feature detection [26], local descriptor extraction [25], quantization with a visual codebook [41], typically created with k-means, assignment of descriptors to visual words and finally descriptor aggregation in a single embedding [19, 32] or individual”
Mining on Manifolds: Metric Learning Without Labels (2018, unknown): “Traditional instance search algorithms use hand-designed descriptors [51, 39, 57, 25], but recent advances show the interest of feature learning [42, 15].”
Neural Codes for Image Retrieval (2014, background): “The performance of adapted L(6)(I) features on the Holidays dataset is better then for previously published systems based on holistic features (unless much higher dimensionality as in [22] is considered).”
UvA-DARE (Digital Academic Repository) Siamese Instance Search for Tracking (2016, background): “The most popular paradigm is based on matching local image descriptors between the query and the candidate image [40, 36, 20, 37, 45, 43] and is especially accurate for buildings [36].”
Aggregating Local Deep Features for Image Retrieval (2015, background): “This becomes a source of accidental false positive matches between local features, and, if the dataset is big enough, between images (as the similarity between resulting global descriptors is aggregated from similarities between pairs of local features [3, 25]).”
Asymmetric metric learning for knowledge transfer (2020, methodology): “Image retrieval Instance-level image retrieval, either using local features [66] or global pooling [36], has relied on SIFT descriptors [43] for more than a decade.”
Exemplar SVMs as visual feature encoders (2015, methodology): “[35], wherein a kernel similar to the popular Hellinger kernel is shown to provide a large improvement in the image retrieval task.”
Siamese Instance Search for Tracking (2016, background): “The most popular paradigm is based on matching local image descriptors between the query and the candidate image [40, 36, 20, 37, 45, 43] and is especially accurate for buildings [36].”
Online Bag-of-Visual-Words Generation for Unsupervised Representation Learning (2020, background): “Thanks to that, they were used extensively in the past [13, 42, 56, 63, 68] and continue to be a key ingredient of several recent deep learning approaches [2, 26, 29, 40].”
Learning Representations by Predicting Bags of Visual Words (2020, background): “BoW have been studied extensively and leveraged in numerous tasks, while multiple extensions [32, 50] and theoretical interpretations [60] have been proposed.”
Aggregating Local Deep Features for Image Retrieval (2015, background): “This becomes a source of accidental false positive matches between local features, and, if the dataset is big enough, between images (as the similarity between resulting global descriptors is aggregated from similarities between pairs of local features [3, 25]).”
Visual Instance Retrieval with Deep Convolutional Networks (2014, background): “The performance of 27) and 28) have been among the state-of-the-art in this domain.”