Semantic part segmentation using compositional model combining shape and appearance

By Alan L. Yuille. 2014

111 citations recorded by the source corpus.

In this paper, we study the problem of semantic part segmentation for animals. This is more challenging than standard object detection, object segmentation and pose estimation tasks because semantic parts of animals often have similar appearance and highly varying shapes. To tackle these challenges, we build a mixture of compositional models to represent the object boundary and the boundaries of semantic parts. And we incorporate edge, appearance, and semantic part cues into the compositional model. Given part-level segmentation annotation, we develop a novel algorithm to learn a mixture of compositional models under various poses and viewpoints for certain animal classes. Furthermore, a linear complexity algorithm is offered for efficient inference of the compositional model using dynamic programming. We evaluate our method for horse and cow using a newly annotated dataset on Pascal VOC 2010 which has pixelwise part labels. Experimental results demonstrate the effectiveness of our method.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

COCO-Stuff: Thing and Stuff Classes in Context (2016, background): “(3) Parts: Thing classes have identifiable parts [56, 19], whereas stuff classes do not (e.”
Toward an Integration of Deep Learning and Neuroscience (2016, background): “Compositional generative models generate a scene from discrete combinations of template parts and their transformations (Wang and Yuille, 2014), in eﬀect performing a rendering of a scene based on its structural description.”
Joint Object and Part Segmentation Using Deep Learned Potentials (2015, methodology): “1 provides the results of SPS [39], HC∗ [21] and ours over semantic part and figure-ground segmentation.”
Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing (2017, background): “Besides human parsing, there has also been increasing research interest on the part segmentation of other objects such as animals or cars [26, 27, 22].”
Deep Feature Factorization For Concept Discovery (2018, methodology): “Since we have not come across examples of part co-segmentation in the literature, we compare against a method for supervised part segmentation, namely Wang et al. [35] (Table 3 in section 4.5).”
Zoom Better to See Clearer: Human Part Segmentation with Auto Zoom Net (2015, methodology): “In addition, human parsing can be widely applicable to other computer vision tasks, e.g., segmentation (Eslami & Williams, 2012; Wang et al., 2015a), pose estimation (Dong et al., 2014), fine-grained recognition (Zhang et al., 2014), and industry applications such as robotics and image descriptions…”
Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery (2021, background): “Other work tackles part segmentation from a single image [47, 43, 19, 22, 1, 23] or point clouds [36, 37, 48, 18].”
Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net (2015, methodology): “In addtion, we also perform parsing experiments on the horse-cow dataset [29], which contains animal instances in a rough bounding box.”
Semantic Object Parsing with Local-Global Long Short-Term Memory (2015, methodology): “The Horse-Cow parsing dataset is a part segmentation benchmark introduced in [31].”
Semantic Object Parsing with Graph LSTM (2016, methodology): “Horse-Cow Parsing dataset [21]: Table 2 shows the comparison results with five state-of-the-art methods on the overall metrics.”
Look into Person: Joint Body Parsing & Pose Estimation Network and a New Benchmark (2018, background): “human parsing, there has also been increasing research interest in the part segmentation of other objects, such as animals or cars [24], [41], [42].”
A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs (2017, background): “The RCN formulation opens up compositional models to a wider array of advanced inference and learning algorithms developed in graphical models, potentially leading to improvements that build on their previous successes in real-world scene parsing (56, 57).”
Teaching Compositionality to CNNs (2017, background): “The first flavor focuses on the creation of hierarchical feature representations by means of statistical modeling [8, 27, 48, 47], reusable deformable parts [49, 29], or compositional graph structures [36, 45].”
Gated Feedback Refinement Network for Dense Image Labeling (2017, methodology): “We split the dataset following [29] and obtain 294 training images and 227 test images.”