Towards Interpretable Object Detection by Unfolding Latent Structures

By Tianfu Wu, Xi Song. 2019

28 citations recorded by the source corpus.

This paper first proposes a method of formulating model interpretability in visual understanding tasks based on the idea of unfolding latent structures. It then presents a case study in object detection using popular two-stage region-based convolutional network (i.e., R-CNN) detection systems. The proposed method focuses on weakly-supervised extractive rationale generation, that is learning to unfold latent discriminative part configurations of object instances automatically and simultaneously in detection without using any supervision for part configurations. It utilizes a top-down hierarchical and compositional grammar model embedded in a directed acyclic AND-OR Graph (AOG) to explore and unfold the space of latent part configurations of regions of interest (RoIs). It presents an AOGParsing operator that seamlessly integrates with the RoIPooling/RoIAlign operator widely used in R-CNN and is trained end-to-end. In object detection, a bounding box is interpreted by the best parse tree derived from the AOG on-the-fly, which is treated as the qualitatively extractive rationale generated for interpreting detection. In experiments, Faster R-CNN is used to test the proposed method on the PASCAL VOC 2007 and the COCO 2017 object detection datasets. The experimental results show that the proposed method can compute promising latent structures without hurting the performance. The code and pretrained models are available at https://github.com/iVMCL/iRCNN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Ethical Implementation of Artificial Intelligence to Select Embryos in In Vitro Fertilization (2021, background): “By forcing the network to reason logically, we may be better able to understand its reasoning process (e.g., Wu and Song [63], Li, Song and Wu [64]).”
Concept whitening for interpretable image recognition (2020, background): “…case-based reasoning (Chen et al., 2018; Li et al., 2018), to incorporate logical or grammatical structures (Li et al., 2017; Granmo et al., 2019; Wu and Song, 2019), to do clas-siﬁcation based on hard attention (Mnih et al., 2014; Ba et al., 2014; Sermanet et al., 2014; Elsayed et al., 2019),…”