Improving Spatial Support for Objects via Multiple Segmentations

By Tomasz Malisiewicz, Alexei A. Efros. 2007

342 citations recorded by the source corpus.

Sliding window scanning is the dominant paradigm in object recognition research today. But while much success has been reported in detecting several rectangular-shaped object classes (i.e. faces, cars, pedestrians), results have been much less impressive for more general types of objects. Several researchers have advocated the use of image segmentation as a way to get a better spatial support for objects. In this paper, our aim is to address this issue by studying the following two questions: 1) how important is good spatial support for recognition? 2) can segmentation provide better spatial support for objects? To answer the first, we compare recognition performance using ground-truth segmentation vs. bounding boxes. To answer the second, we use the multiple segmentation approach to evaluate how close can real segments approach the ground-truth for real objects, and at what cost. Our results demonstrate the importance of finding the right spatial support for objects, and the feasibility of doing so without excessive computational burden.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

What Makes for Effective Detection Proposals? (2015, background): “We can then write: AR = 2 1ˆ 0.5 recall(o) do = 2 n n∑ i=1 max ( IoU(gti) − 0.5, 0 ) which is the same as the average best overlap (ABO) [19] or the average best spatial support (BSS) [63] truncated at 0.5 IoU.”
Learning visual groups from co-occurrences in space and time (2015, background): “…showing up in the tasks of edge/contour detection Canny (1986); Arbelaez et al. (2011); Isola et al. (2014) , (semantic) segmentation Shi & Malik (2000); Malisiewicz & Efros (2007), and object proposals Alexe et al. (2012); Zitnick & Doll ´ ar (2014); Krahnenbuhl & Koltun (2015), among others.”
Object categorization using co-occurrence, location and appearance (2008, methodology): “More details of combining stable segmentations with BoF can be found in [18].”
Recognition by association via learning per-exemplar distances (2008, methodology): “We use the multiple segmentation approach [16, 12] – which was shown to provide good spatial support for many different object types – to generate a large collection of candidate segments.”
Probabilistic Joint Image Segmentation and Labeling (2011, background): “However, most existing mid-level segmentation algorithms cannot generate one unique, yet accurate segmentation per image, across mul tiple images, for the same set of generic parameters [9, 10].”
Contour Detection and Hierarchical Image Segmentation (2011, methodology): “We evaluate the performance using the ground-truth object instance labeling of [11], which is cleaner and more precise than the original data.”
Multi-task low-rank affinity pursuit for image segmentation (2011, methodology): “Here we use the cleaned up groundtruth object instance labeling [22], which is cleaner and more precise than the original data.”
Object Recognition by Sequential Figure-Ground Ranking (2011, background): “Malisiewicz and Efros (2008) generate a large pool of segments (Malisiewicz and Efros 2007) and recognize them using a nearest-neighbor classifier based on learned distance functions.”
From contours to regions: An empirical evaluation (2009, background): “Applications such as object recognition [34] [25] [1] [19] and monocular inference of 3D structure [20] [41] have led to a renewed interest in algorithms for automatic segmentation of an image into closed regions.”
CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts (2012, methodology): “By generating tens to hundreds of thousands of segments per image, Malisiewicz and Efros [26] produced very good quality regions for the MSRC data set by merging pairs and triplets of segments obtained using the Mean Shift [27], Normalized Cuts [28], and Felzenszwalb-Huttenlocher’s (FH) [29] algorit”
BiCoS: A Bi-level co-segmentation method for image classification (2011, background): “There is abundant evidence that accurate foreground-background segmentation can benefit visual recognition [24].”
Learning full pairwise affinities for spectral segmentation (2010, methodology): “This measure has been used for the evaluation of the pixel-wise classification task in recognition [39].”
Constrained parametric min-cuts for automatic object segmentation (2010, methodology): “By generating tens to hundreds of thousand of segments per image, Malisiewicz and Efros [21] produce very good quality segments on the MSRC dataset by merging pairs and triplets of segments obtained using the Mean Shift [9], Normalized Cuts [31] and Felzenszwalb-Huttenlocher's (FH) [12] algorithms.”
Pylon Model for Semantic Segmentation (2011, background): “This is, in fact, a manifestation of a well-known chicken-and-egg problem between segmentation and recognition (given spatial support based on proper segmentation, recognition is easy [20], but to get the proper segmentation prior recognition is needed).”
Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation (2012, methodology): “We first evaluate our approach on MSRC-21 [22], which will be referred to as origMSRC, as well as on the more accurately annotated version [19], referred to as cleanMSRC.”
Image Segmentation by Cascaded Region Agglomeration (2013, methodology): “For ground truth images, we use the cleaner and precise version of [15]; avg.”
Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Exploring Tiny Images: the Roles of Appearance and Contextual Information for Machine and Human Object Recognition (positive): “We approach this task at the region or segment level since good spatial support is shown to significantly help recognition [43], [44].”
Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation (2017, background): “With respect to the combinatorial approaches of [4], [7], our main contribution is to develop efficient algorithms to explore a much larger combinatorial space by taking into account a set of object examples, increasing the likelihood of having complete objects in the pool of proposals.”
Instance Segmentation of Indoor Scenes Using a Coverage Loss (2014, background): “Motivated by the observation that a single segmentation of an image is unlikely to produce a perfect result, numerous approaches [13] [14] [15] [16] make use of multiple segmentations of an image.”