Controlling explanatory heatmap resolution and semantics via decomposition depth

By Sebastian Bach, Alexander Binder, K. Müller, Wojciech Samek. 2016

25 citations recorded by the source corpus.

We present an application of the Layer-wise Relevance Propagation (LRP) algorithm to state of the art deep convolutional neural networks and Fisher Vector classifiers to compare the image perception and prediction strategies of both classifiers with the use of visualized heatmaps. Layer-wise Relevance Propagation (LRP) is a method to compute scores for individual components of an input image, denoting their contribution to the prediction of the classifier for one particular test point. We demonstrate the impact of different choices of decomposition cut-off points during the LRP-process, controlling the resolution and semantics of the heatmap on test images from the PASCAL VOC 2007 test data set.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “[93]–[95]).”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, neutral): “-rule Ri←j = zij σ( ∑ i zij) Rj - FV top layer in [34,140]”
Understanding and Comparing Deep Neural Networks for Age and Gender Classification (2017, methodology): “We follow [4] in distributing Rj for all neurons of some of the lower layers uniformly across their respective inputs, such that the granularity of the visualizations for all models are comparable.”
Towards multi-modal causability with Graph Neural Networks enabling information fusion for explainable AI (2021, methodology): “The relevance of each pixel can eventually be stored in a heatmap [79] denoted by 𝑹 ( 𝒙 ) = { 𝑅 𝑝 ( 𝒙 )} , which is defined as the sum of the relevances in the pixel space corresponding to the total relevance detected by the model, i.e.”