Fisher and VLAD with FLAIR

By K. V. D. Sande, Cees G. M. Snoek, Arnold M. W. Smeulders. 2014

71 citations recorded by the source corpus.

A major computational bottleneck in many current algorithms is the evaluation of arbitrary boxes. Dense local analysis and powerful bag-of-word encodings, such as Fisher vectors and VLAD, lead to improved accuracy at the expense of increased computation time. Where a simplification in the representation is tempting, we exploit novel representations while maintaining accuracy. We start from state-of-the-art, fast selective search, but our method will apply to any initial box-partitioning. By representing the picture as sparse integral images, one per codeword, we achieve a Fast Local Area Independent Representation. FLAIR allows for very fast evaluation of any box encoding and still enables spatial pooling. In FLAIR we achieve exact VLAD's difference coding, even with L2 and power-norms. Finally, by multiple codeword assignments, we achieve exact and approximate Fisher vectors with FLAIR. The results are a 18x speedup, which enables us to set a new state-of-the-art on the challenging 2010 PASCAL VOC objects and the fine-grained categorization of the CUB-2011 200 bird species. Plus, we rank number one in the official ImageNet 2013 detection challenge.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiaogang Wang
2. Trevor Darrell
3. Li Fei-Fei
4. Bernt Schiele
5. Wanli Ouyang
6. Alan L. Yuille
7. Luc Van Gool
8. Chen Change Loy

In Computer Vision and Pattern Recognition

1. Xiaogang Wang
2. Luc Van Gool
3. Dacheng Tao
4. Ming-Hsuan Yang
5. Trevor Darrell
6. Yi Yang
7. Andrew Zisserman
8. Andrea Vedaldi

In Media Technology

1. Chen Change Loy
2. Li Fei-Fei
3. Alexander C. Berg
4. Jitendra Malik
5. Trevor Darrell
6. Xiaoou Tang
7. Jia Deng
8. Luc Van Gool

Citations in the corpus

• ImageNet Large Scale Visual Recognition Challenge (2014, methodology): “…Euvision Technologies Koen E. A. van de Sande, Daniel H. F. Fontijne, Cees G. M. Snoek, Harro M. G. Stokman, Arnold W. M. Smeulders (van de Sande et al., 2014) VGG 15.2 46.4 - Visual Geometry Group, University of Oxford Karen Simonyan, Andrea Vedaldi, Andrew Zisserman (Simonyan et al.,…”
• Particular object retrieval with integral max-pooling of CNN activations (2015, methodology): “Integral images facilitate the evaluation of many region candidates (Uijlings et al., 2013) based on VLAD or Fisher vectors (Van de Sande et al., 2014).”
• Best of both worlds: Human-machine collaboration for object annotation (2015, background): “The field of large-scale object detection has leaped forward in the past few years [20, 43, 11, 45, 61, 24, 53], with significant progress both in techniques [20, 43, 53, 45] as well as scale: hundreds of thousands of object detectors can now be trained directly from web data [8, 15, 11].”
• DeepID-Net: Object Detection with Deformable Part Based Convolutional Neural Networks (2017, methodology): “approach Flair RCNN Berkeley Vision UvA-Euvision DeepInsight GoogLeNet Superpixel Label ours [62] [16] [16] [55] [67] [60] [68] ImageNet val2 (avg) n/a n/a n/a n/a 42 44.”
• Region-Based Convolutional Networks for Accurate Object Detection and Segmentation (2016, methodology): “Notably, UvA-Euvision's entry did not use CNNs and was based on a fast VLAD encoding [60].”
• DeepID-Net: Deformable deep convolutional neural networks for object detection (2014, methodology): “Table 2 shows the experimental results on VOC-2007 testing data, which include approaches using hand-crafted features [15, 33, 47, 46, 11], deep CNN features [14, 19], and CNN features with deformation learning [16].”
• Supervised mid-level features for word image representation (2014, methodology): “Van de Sande et al have recently proposed FLAIR, an approach to efficiently compute region-based Fisher vectors [36].”
• Spatio-temporal Object Detection Proposals (2014, unknown): “Some recent work partially addresses this issue using efficient implementations [23,40] or approximate normalizations [29].”
• Factors in Finetuning Deep Model for Object Detection with Long-Tail Distribution (2016, methodology): “For detection approaches using hand-crafted features [10, 36], feature extraction is separated from the classifier learning task.”
• Predicting Good Features for Image Geo-Localization Using Per-Bundle VLAD (2015, unknown): “3.2).”
• Learning Deep Representation with Large-Scale Attributes (2015, methodology): “Many existing works are included for comparison [12, 30, 40, 38, 10, 11, 14, 13].”