TreeView: Peeking into Deep Neural Networks Via Feature-Space Partitioning

By Jayaraman J. Thiagarajan, Bhavya Kailkhura, Prasanna Sattigeri, Karthikeyan Natesan Ramamurthy. 2016

44 citations recorded by the source corpus.

With the advent of highly predictive but opaque deep learning models, it has become more important than ever to understand and explain the predictions of such models. Existing approaches define interpretability as the inverse of complexity and achieve interpretability at the cost of accuracy. This introduces a risk of producing interpretable but misleading explanations. As humans, we are prone to engage in this kind of behavior \cite{mythos}. In this paper, we take a step in the direction of tackling the problem of interpretability without compromising the model accuracy. We propose to build a Treeview representation of the complex model via hierarchical partitioning of the feature space, which reveals the iterative rejection of unlikely class labels until the correct association is predicted.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dino Pedreschi
2. Riccardo Guidotti
3. Fosca Giannotti
4. Anna Monreale
5. Francisco Herrera
6. Longo Luca
7. Giulia Vilone
8. Franco Turini

In Computer Vision and Pattern Recognition

1. Salvador Garcia
2. Francisco Herrera
3. Longo Luca
4. Giulia Vilone
5. Mohammed Berrada
6. Amina Adadi
7. Adrien Bennetot
8. Alejandro Barredo Arrieta

In Safety, Robustness & Interpretability

1. Dino Pedreschi
2. Riccardo Guidotti
3. Longo Luca
4. Giulia Vilone
5. Fosca Giannotti
6. Anna Monreale
7. Franco Turini
8. Mohammed Berrada

Citations in the corpus

• Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “In the same direction, the authors in [135] propose a hierarchical partitioning of the feature space that reveals the iterative rejection of unlikely class labels, until association is predicted.”
• Generative Counterfactual Introspection for Explainable Deep Learning (2019, methodology): “Alternatively, we can build a simpler localized model to approximate the complex nonlinear model [14], [15].”
• Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “[107] proposed an approach for building TreeView visualizations using a surrogate model.”
• A Survey of Methods for Explaining Black Box Models (2018, methodology): “Reference [119] shows the extraction of a visual interpretation r of a DNN using a decision tree.”
• Explainable Artificial Intelligence: a Systematic Review (2020, methodology): “Finally, TreeView [219] consists of a scatter plot representation of a DNN via hierarchical partitioning of the feature space.”
• Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey (2019, methodology): “PDA [84] image feature importance local, ante-hoc DVE [83] image visualization local, post-hoc Grad-CAM [76] image saliency mask local, post-hoc Grad-CAM++ [78] image saliency mask local, post-hoc Smooth-Grad [86] image sensitivity analysis local, ante-hoc ME [88] image visualization local, post-hoc”