Particular object retrieval with integral max-pooling of CNN activations

By Giorgos Tolias, Ronan Sicre, Herv'e J'egou. 2015

1,023 citations recorded by the source corpus.

Recently, image representation built upon Convolutional Neural Network (CNN) has been shown to provide effective descriptors for image search, outperforming pre-CNN features as short-vector representations. Yet such models are not compatible with geometry-aware re-ranking methods and still outperformed, on some particular object retrieval benchmarks, by traditional image search systems relying on precise descriptor matching, geometric re-ranking, or query expansion. This work revisits both retrieval stages, namely initial search and re-ranking, by employing the same primitive information derived from the CNN. We build compact feature vectors that encode several image regions without the need to feed multiple inputs to the network. Furthermore, we extend integral images to handle max-pooling on convolutional layer activations, allowing us to efficiently localize matching objects. The resulting bounding box is finally used for image re-ranking. As a result, this paper significantly improves existing CNN-based recognition pipeline: We report for the first time results competing with traditional methods on the challenging Oxford5k and Paris6k datasets.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

Citations in the corpus

Revisiting IM2GPS in the Deep Learning Era (2017, methodology): “We are studying image retrieval geolocalization which has overlap with instance-level scene retrieval [33, 2, 23, 7]; Since this line of work mostly focuses on instance-level matching, benchmarks designed for this task consist of popular scenes or landmarks [21, 22, 12] and similarity between…”
Emerging Properties in Self-Supervised Vision Transformers (2021, neutral): “DINO ViT features trained on GLDv2 are remarkably good, outperforming previously published methods based on off-the-shelf descriptors [55, 46].”
Deep Clustering for Unsupervised Learning of Visual Features (2018, methodology): “We apply R-MAC with a resolution of 1024 pixels and 3 grid levels [59]”
NetVLAD: CNN Architecture for Weakly Supervised Place Recognition (2015, background): “[45] pool at different scales, while Kalantidis et al.”
MultiGrain: a unified image embedding for classes and instances (2019, background): “For instance, a competitive baseline for instance retrieval on various datasets is the R-MAC image descriptor [43].”
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection (2017, methodology): “Early approaches, such as [6], heavily depend on extracting local features such as SIFT [30], SURF [3], MSER [32] and matching them to retrieve instances [29, 48].”
End-to-End Learning of Deep Visual Representations for Image Retrieval (2016, methodology): “For the representation, we build on the regional maximumactivations of convolutions (R-MAC) descriptor (Tolias et al. 2016).”
Fine-Tuning CNN Image Retrieval with No Human Annotation (2017, methodology): “successfully perform image search [8], [9], [10], [11], [12].”
Supplementary Material for Unsupervised Pre-Training of Image Features on Non-Curated Data (2019, methodology): “We apply R-MAC with a resolution of 1024 pixels and 3 grid levels [6].”
A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, neutral): “Several deep learning models have been investigated for object retrieval, such as integral image driven max-pooling on CNN activations [214], pooling of the relevant features based on the region proposal network [65], replicator equation based simultaneous selection and weighting of the primitive…”
Task-Oriented Multi-User Semantic Communications (2021, background): “The deep network architectures include single feedforward pass models [26], multiple feedforward pass models [27], attention based models [28], and deep hashing embedding based models [29].”
Unifying Deep Local and Global Features for Image Search (2020, methodology): “Today, most high-performing global features are based on deep convolutional neural networks [5,4,58,1,17,46,47], which are trained with rankingbased [9,50,19] or classification losses [60,11].”
Unsupervised Pre-Training of Image Features on Non-Curated Data (2019, neutral): “[45]: we apply R-MAC with a resolution of 1024 pixels and 3 grid levels and we report mAP on instance-level image retrieval on Oxford Build-ings [38] and Paris [39] datasets.”
Learning and aggregating deep local descriptors for instance-level recognition (2020, methodology): “This approach is applied to activations generated by pretrained networks [4,45,23] or end-to-end learned networks [35,14,15].”
Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking (2018, methodology): “We consider max-pooling (MAC) [38, 48], sum-pooling (SPoC) [3], weighted sum-pooling (CroW) [20], regional max-pooling (R-MAC) [48], generalized mean-pooling (GeM) [37], and NetVLAD pooling [1].”