Towards Interpretable R-CNN by Unfolding Latent Structures

By Tianfu Wu, Xilai Li, Xi Song, Bo Li. 2017

20 citations recorded by the source corpus.

This paper first proposes a method of formulating model interpretability in visual understanding tasks based on the idea of unfolding latent structures. It then presents a case study in object detection using popular two-stage region-based convolutional network (i.e., R-CNN) detection systems. We focus on weakly-supervised extractive rationale generation, that is learning to unfold latent discriminative part configurations of object instances automatically and simultaneously in detection without using any supervision for part configurations. We utilize a top-down hierarchical and compositional grammar model embedded in a directed acyclic AND-OR Graph (AOG) to explore and unfold the space of latent part configurations of regions of interest (RoIs). We propose an AOGParsing operator to substitute the RoIPooling operator widely used in R-CNN. In detection, a bounding box is interpreted by the best parse tree derived from the AOG on-the-fly, which is treated as the qualitatively extractive rationale generated for interpreting detection. We propose a folding-unfolding method to train the AOG and convolutional networks end-to-end. In experiments, we build on R-FCN and test our method on the PASCAL VOC 2007 and 2012 datasets. We show that the method can unfold promising latent structures without hurting the performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “…as part of the loss function (Dong et al., 2017b,a), modules that add additional capabilities (Santoro et al., 2017; Palm et al., 2017), or as part of the architecture structure, in terms of operations between layers (Li et al., 2017; Wu et al., 2017; Louizos et al., 2017; Goudet et al., 2017).”
Visual interpretability for deep learning: a survey (2018, methodology): “[Wu et al. , 2017] utilizes a top-down hierarchical and compositional grammar, namely an And-Or graph (AOG), to model latent conﬁgurations of object parts.”
Explainability in Deep Reinforcement Learning (2020, background): “(interpretable) representations of the conv-layers of these networks [114, 113], as well as end-to-end learning of interpretable networks, working directly with comprehensible patterns, which are also a trending angle [108].”