Interactively Transferring CNN Patterns for Part Localization

By Quanshi Zhang, Ruiming Cao, Shengming Zhang, Mark Edmonds, Ying Nian Wu, Song-Chun Zhu. 2017

16 citations recorded by the source corpus.

In the scenario of one/multi-shot learning, conventional end-to-end learning strategies without sufficient supervision are usually not powerful enough to learn correct patterns from noisy signals. Thus, given a CNN pre-trained for object classification, this paper proposes a method that first summarizes the knowledge hidden inside the CNN into a dictionary of latent activation patterns, and then builds a new model for part localization by manually assembling latent patterns related to the target part via human interactions. We use very few (e.g., three) annotations of a semantic object part to retrieve certain latent patterns from conv-layers to represent the target part. We then visualize these latent patterns and ask users to further remove incorrect patterns, in order to refine part representation. With the guidance of human interactions, our method exhibited superior performance of part localization in experiments.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Visual interpretability for deep learning: a survey (2018, methodology): “Section 7 introduces methods to learn new models via human-computer interactions [Zhang et al., 2017b] and active question-answering with very limited human supervision [Zhang et al.”
Examining CNN representations with respect to Dataset Bias (2017, methodology): “Methods of (Bansal, Farhadi, and Parikh 2014; Zhang et al. 2014) computed the distributions of a CNN’s prediction errors among testing samples, in order to summarize failure modes of the CNN.”
Interpreting CNN knowledge via an Explanatory Graph (2017, methodology): “mage I. Experimental settings of three-shot learning We learned the explanatory graph based on a ﬁne-tuned VGG16 network and built the AOG following the scenario of multi-shot learning introduced in (Zhang et al. 2017a). For each category, we used three annotations of the head part to learn three he”