Unsupervised learning of object semantic parts from internal states of CNNs by population encoding

By Zhishuai Zhang, Cihang Xie, Vittal Premachandran, Alan L. Yuille. 2015

44 citations recorded by the source corpus.

We address the key question of how object part representations can be found from the internal states of CNNs that are trained for high-level tasks, such as object classification. This work provides a new unsupervised method to learn semantic parts and gives new understanding of the internal representations of CNNs. Our technique is based on the hypothesis that semantic parts are represented by populations of neurons rather than by single filters. We propose a clustering technique to extract part representations, which we call Visual Concepts. We show that visual concepts are semantically coherent in that they represent semantic parts, and visually coherent in that corresponding image patches appear very similar. Also, visual concepts provide full spatial coverage of the parts of an object, rather than a few sparse parts as is typically found in keypoint annotations. Furthermore, We treat single visual concept as part detector and evaluate it for keypoint detection using the PASCAL3D+ dataset and for part detection using our newly annotated ImageNetPart dataset. The experiments demonstrate that visual concepts can be used to detect parts. We also show that some visual concepts respond to several semantic parts, provided these parts are visually similar. Thus visual concepts have the essential properties: semantic meaning and detection capability. Note that our ImageNetPart dataset gives rich part annotations which cover the whole object, making it useful for other part-related applications.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Mitigating adversarial effects through randomization (2017, background): “…2015; He et al., 2016a; Huang et al., 2016), object detection (Girshick et al., 2014; Girshick, 2015; Ren et al., 2015; Dai et al., 2016), semantic segmentation (Long et al., 2015; Chen et al., 2017; Zheng et al., 2015), visual concept discovery (Wang et al., 2015; Zhang et al., 2017) etc.”
Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks (2018, methodology): “Our novelty lies in outlining methods that go beyond simply demonstrating that multiple ﬁlters better encode concepts that single ones [1, 16] to quantifying and describing how a concept is encoded.”
Compositional Convolutional Neural Networks: A Deep Architecture With Innate Robustness to Partial Occlusion (2020, methodology): “For evaluation we use the Occluded-Vehicles dataset as proposed in [24] and extended in [11].”
DeepVoting: An Explainable Framework for Semantic Part Detection under Partial Occlusion (2017, methodology): “On the other hand, semantic part detection, despite its importance, has been much less studied (Wang et al. 2015a).”
Visual Concepts and Compositional Voting (2017, methodology): “Our longer report [40] gives results for different variants of visual concepts (e.”
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition Under Occlusion (2020, unknown): “This property was also observed in a number of related works that used clustering in the neural feature space (Wang et al. 2015; Liao et al. 2016; Wang et al. 2017).”