Semantic segmentation using regions and parts

By Pablo Arbeláez, Bharath Hariharan, Chunhui Gu, Saurabh Gupta, Lubomir D. Bourdev, Jitendra Malik. 2012

325 citations recorded by the source corpus.

We address the problem of segmenting and recognizing objects in real world images, focusing on challenging articulated categories such as humans and other animals. For this purpose, we propose a novel design for region-based object detectors that integrates efficiently top-down information from scanning-windows part models and global appearance cues. Our detectors produce class-specific scores for bottom-up regions, and then aggregate the votes of multiple overlapping candidates through pixel classification. We evaluate our approach on the PASCAL segmentation challenge, and report competitive performance with respect to current leading techniques. On VOC2010, our method obtains the best results in 6/20 categories and the highest performance on articulated objects.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation (2013, methodology): “This recall is notably lower than in PASCAL, where it is approximately 98%, indicating significant room for improvement in the region proposal stage.”
Simultaneous Detection and Segmentation (2014, methodology): “[2] use poselet detections [4] as features to score region candidates, in addition to appearance-based cues.”
Learning Open-World Object Proposals without Learning to Classify (2021, background): “They have become the integral pre-processing steps for many computer vision systems, including object detection [33,41,42,45,59,74], segmentation [3,8,13], object discovery [12, 16, 47], weakly supervised object detection [5, 21, 53], visual tracking [35, 63], content-aware retargeting [51], etc.”
Object-Proposal Evaluation Protocol is ‘Gameable’ (2015, background): “Types of Object Proposals: Object proposals can be broadly categorized into two categories: • Window scoring : In these methods, the space of all possible windows in an image is sampled to get a sub-set of the windows ( e.g ., via sliding window).”
Conditional Random Fields as Recurrent Neural Networks (2015, background): “The first strategy is based on utilizing separated mechanisms for both feature extraction, and image segmentation exploiting the edges of the image [2, 20].”
Feedforward semantic segmentation with zoom-out features (2014, background): “Another recent trend has been to follow a multi-stage approach: First a set of proposal regions is generated, by a category-independent [6, 38] or category-aware [2] mechanism.”
Object-Contextual Representations for Semantic Segmentation (2019, methodology): “tic meaning. Our approach empirically outperforms all the previous methods by a large margin on various benchmarks. Region-based segmentation. Before the era of deep learning, region-based approaches [1,2,17,16] have been used to address the scene understanding problem. According to [2], the convent”
Iterative Instance Segmentation (2015, background): “[1] paste figureground masks for poselet detections [4].”
Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation (2015, methodology): “In [8], another step was taken by considering hierarchical segmentations at three different scales and combining pairs and triplets of adjacent regions from the two coarser scales to produce object proposals.”
Crisp Boundary Detection Using Pointwise Mutual Information (2014, methodology): “Object boundary cues have been used to aid in segmentation [1,2,3], object detection and recognition [4,5], and recovery of intrinsic scene properties such as shape, reflectance, and illumination [6].”
GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, methodology): “This pipeline has been successfully applied in image segmentation from superpixels [60], constructing region proposals for object detection [75, 94] and semantic segmentation [3].”
Region-Based Convolutional Networks for Accurate Object Detection and Segmentation (2016, methodology): “We follow [59], [68] and extend the PASCAL segmentation training set to include the extra annotations made available by Hariharan et al.”
Inferring 3D Object Pose in RGB-D Images (2015, methodology): “When we use this 3D bounding box for 3D detection we observe that we are able to outperform the current state-of-the-art method [33] by 19% absolute AP points (48% relative), while at the same time being at least an order of magnitude faster.”
Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation (2017, methodology): “Recently, this approach has dominated the Pascal segmentation challenge [6], [7], [8], improved object detection [9], fine-grained categorization [10] and proven competitive in large-scale classification [11].”
Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling (2013, background): “The conditional random field [16] is a powerful model for structured output prediction (such as sequence prediction, text parsing, and image segmentation), and has been widely used in computer vision [8, 3, 4, 7].”
Learning Discriminative Part Detectors for Image Classification and Cosegmentation (2013, unknown): “They represent objects or images by a set of important parts, and achieve state-of-the-art results for object detection [4, 15], action recognition [39], segmentation [2], etc. Learning these models has, however, been a challenge.”
Beyond pixels: A comprehensive survey from bottom-up to semantic image segmentation and cosegmentation (2015, unknown): “Arbelaez et al. [118] used the Poselet detector to segment articulated objects.”
Multi-instance object segmentation with occlusion handling (2015, methodology): “Semantic segmentation [3, 6, 28, 16, 31] assigns a category label to each segment generated by a bottom-up segmentation algorithm.”
Deep Learning for Generic Object Detection: A Survey (2018, background): “A comprehensive review of object proposal algorithms is outside the scope of this paper, because object proposals have applications beyond object detection [6, 72, 252].”
BING: Binarized normed gradients for objectness estimation at 300fps (2014, unknown): “A generic objectness measure has great potential to be used as a pre-filter for many vision tasks, including object detection [3–5], visual tracking [6, 7], object discovery [8, 9], semantic segmentation [10, 11], content aware image retargeting [12], and action recognition [13].”