SIFT Meets CNN: A Decade Survey of Instance Retrieval

By Liang Zheng, Yi Yang, Qi Tian. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016

750 citations recorded by the source corpus.

In the early days, content-based image retrieval (CBIR) was studied with global features. Since 2003, image retrieval based on local descriptors (de facto SIFT) has been extensively studied for over a decade due to the advantage of SIFT in dealing with image transformations. Recently, image representations based on the convolutional neural network (CNN) have attracted increasing interest in the community and demonstrated impressive performance. Given this time of rapid evolution, this article provides a comprehensive survey of instance retrieval over the last decade. Two broad categories, SIFT-based and CNN-based methods, are presented. For the former, according to the codebook size, we organize the literature into using large/medium-sized/small codebooks. For the latter, we discuss three lines of methods, i.e., using pre-trained or fine-tuned CNN models, and hybrid methods. The first two perform a single-pass of an image to the network, while the last category employs a patch-based feature extraction scheme. This survey presents milestones in modern instance retrieval, reviews a broad selection of previous works in different categories, and provides insights on the connection between SIFT and CNN-based methods. After analyzing and comparing retrieval performance of different categories on several datasets, we discuss promising directions towards generic and specialized instance retrieval.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6d902439b736a7546dd8872b307fb760087ca629

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Dynamic Match Kernel With Deep Convolutional Features for Image Retrieval (2018, methodology): “Each of them is further arranged based on the implementation details [22].”
A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, background): “A huge progress has been made in this decade to utilize the power of deep learning for content based image retrieval [46], [47], [48], [49].”
From BoW to CNN: Two Decades of Texture Representation for Texture Classification (2018, background): “…applications of texture analysis include medical image analysis [48, 158, 177], quality inspection [250], content based image retrieval [148, 216, 266], analysis of satellite or aerial imagery [96, 83], face analysis [4, 49, 215, 265], biometrics [137, 185], object recognition [210, 173, 263],…”
T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval (2021, positive): “VLAD [15] and NetVLAD [1] have achieved great impacts in aggregating discriminative features for video classification [10, 36], video retrieval [24], person re-identification [40].”
Deep Learning for Free-Hand Sketch: A Survey (2020, background): “FG-SBIR is essentially a kind of instance-level retrieval [197].”
Robust Anchor Embedding for Unsupervised Video Person re-IDentification in the Wild (2018, background): “Person re-identification (re-ID) addresses the problem of searching specific persons across disjoint camera views [54,55].”
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection (2017, methodology): “[55] provides a comprehensive overview of the popular methods in this field.”
Hierarchical Discriminative Learning for Visible Thermal Person Re-Identification (2018, background): “Person re-identification (REID) addresses the problem of matching different persons across disjoint camera views (Zheng, Yang, and Tian 2017; Wang et al. 2016b).”
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro (2017, methodology): “This method assigns a uniform label distribution to the unlabeled images, which regularizes the supervised model and improves the baseline.”
Deep Learning vs. Traditional Computer Vision (2019, methodology): “For example, in a security camera, a CV algorithm can efficiently detect faces or other features [26] or moving objects [27] in the scene.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, background): “Several matching related applications, such as image retrieval (Zheng et al. 2018) and visual localization (Piasco et al. 2018), have also been evaluated recently.”
Large-Scale Image Retrieval with Attentive Deep Local Features (2016, neutral): “See [43] for a recent survey.”
Bi-Directional Center-Constrained Top-Ranking for Visible Thermal Person Re-Identification (2020, background): “PERSON re-identification (re-ID) is a recognition task which aims at matching a specific query person from a set of person images captured by other disjoint surveillance cameras [1].”
Person Re-identification: Past, Present and Future (2016, background): “Person re-ID lies inbetween image classiﬁcation [14] and instance retrieval [19] in terms of the relationship between training and testing classes (Table 1).”
Re-ranking Person Re-identification with k-Reciprocal Encoding (2017, methodology): “Re-ranking methods have been successfully studied to improve object retrieval accuracy [53].”
Unsupervised Person Re-identification (2017, methodology): “Generally speaking, from the perspective of model, classiﬁcation models as used in image classiﬁcation [23] and siamese models that use image pairs [26] or triplets [27] are two types of CNN models that are commonly employed in re-ID.”