The Visual Extent of an Object

By Jasper R. R. Uijlings, Arnold M. W. Smeulders, Remko J. H. Scha. 2011

32 citations recorded by the source corpus.

The visual extent of an object reaches beyond the object itself. This is a long standing fact in psychology and is reflected in image retrieval techniques which aggregate statistics from the whole image in order to identify the object within. However, it is unclear to what degree and how the visual extent of an object affects classification performance. In this paper we investigate the visual extent of an object on the Pascal VOC dataset using a Bag-of-Words implementation with (colour) SIFT descriptors.Our analysis is performed from two angles. (a) Not knowing the object location, we determine where in the image the support for object classification resides. We call this the normal situation. (b) Assuming that the object location is known, we evaluate the relative potential of the object and its surround, and of the object border and object interior. We call this the ideal situation. Our most important discoveries are: (i) Surroundings can adequately distinguish between groups of classes: furniture, animals, and land-vehicles. For distinguishing categories within one group the surroundings become a source of confusion. (ii) The physically rigid plane, bike, bus, car, and train classes are recognised by interior boundaries and shape, not by texture. The non-rigid animals dog, cat, cow, and sheep are recognised primarily by texture, i.e. fur, as their projected shape varies greatly. (iii) We confirm an early observation from human psychology (Biederman in Perceptual Organization, pp. 213–263, 1981): in the ideal situation with known object locations, recognition is no longer improved by considering surroundings. In contrast, in the normal situation with unknown object locations, the surroundings significantly contribute to the recognition of most classes.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Controlling explanatory heatmap resolution and semantics via decomposition depth (2016, background): “for neural networks by highlighting dominant filter activations [1], computing saliency maps visualizing local sensitivities [2] or the identification of support regions [3] critical to the prediction for SVM [4] classifiers with max-pooling feature mapping and the explanation of hard mappings with ”
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation (2015, background): “The first works to our knowledge to visualize BoW models are [33] and [34].”
Analyzing Classifiers: Fisher Vectors and Deep Neural Networks (2015, background): “So do [28] explain predictions for Bag of Word (BoW) features with hard mapping (Vector Quantization) and Histogram Intersection kernels, and [16] identifies image regions critical for the prediction of a linear SVM classifier with max-pooling feature aggregation algorithm.”
Interpretable human action recognition in compressed domain (2017, background): “[14] – describing a system to explain models incorporating vector quantization, histogram intersection kernels and SVMs – or the work of [15] – presenting an algorithm for identifying connected image regions (support regions) critical for the prediction of a linear classifier on top of a max-pooling”