Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval

By Xiu-Shen Wei, Jian-Hao Luo, Jianxin Wu, Zhi-Hua Zhou. IEEE Transactions on Image Processing, 2016

442 citations recorded by the source corpus.

Deep convolutional neural network models pre-trained for the ImageNet classification task have been successfully adopted to tasks in other domains, such as texture description and object proposal generation, but these tasks require annotations for images in the new domain. In this paper, we focus on a novel and challenging task in the pure unsupervised setting: fine-grained image retrieval. Even with image labels, fine-grained images are difficult to classify, letting alone the unsupervised retrieval task. We propose the selective convolutional descriptor aggregation (SCDA) method. The SCDA first localizes the main object in fine-grained images, a step that discards the noisy background and keeps useful deep descriptors. The selected descriptors are then aggregated and the dimensionality is reduced into a short feature vector using the best practices we found. The SCDA is unsupervised, using no image label or bounding box annotation. Experiments on six fine-grained data sets confirm the effectiveness of the SCDA for fine-grained image retrieval. Besides, visualization of the SCDA features shows that they correspond to visual attributes (even subtle ones), which might explain SCDA’s high-mean average precision in fine-grained retrieval. Moreover, on general image retrieval data sets, the SCDA achieves comparable retrieval results with the state-of-the-art general image retrieval approaches.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a7506e8931e3f7ba5446643d6f74245976a3fdae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition (2019, background): “Note that, besides the extreme imbalance, the iNaturalist datasets also face the fine-grained problem [32, 35, 30, 31].”
A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, methodology): “(2017) [199] have proposed a selective convolutional descriptor aggregation (SCDA) to generate the descriptor for image retrieval.”
In Defense of Fully Connected Layers in Visual Representation Transfer (2017, methodology): “For unsupervised ﬁne-grained image retrieval and object localization, we use the SCDA method [17] to show performance of different CNN models.”
Interpreting CNN Models for Apparent Personality Trait Regression (2017, background): “To understand how DAN+ works to predict the personality traits, we make a modification at the last layers of the network, that allows to visualize the regions of the image that support the decision of the network.”
Attention-Based Dropout Layer for Weakly Supervised Object Localization (2019, background): “Existing approaches mine and track discriminative features of each class for object detection [45, 36, 37, 9, 45, 25, 21, 41, 19, 2, 39, 15, 63, 7, 5, 4, 48, 14, 65, 32, 31, 58, 62, 8, 6] and segmentation [33, 29, 18, 16, 24, 52, 50].”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “, the weakly-supervised localization module proposed in [170], which is in turn inspired by [32].”