Explaining the Unexplained: A CLass-Enhanced Attentive Response (CLEAR) Approach to Understanding Deep Neural Networks

By Devinder Kumar, Alexander Wong, Graham W. Taylor. 2017

63 citations recorded by the source corpus.

In this work, we propose CLass-Enhanced Attentive Response (CLEAR): an approach to visualize and understand the decisions made by deep neural networks (DNNs) given a specific input. CLEAR facilitates the visualization of attentive regions and levels of interest of DNNs during the decision-making process. It also enables the visualization of the most dominant classes associated with these attentive regions of interest. As such, CLEAR can mitigate some of the shortcomings of heatmap-based methods associated with decision ambiguity, and allows for better insights into the decision-making process of DNNs. Quantitative and qualitative experiments across three different datasets demonstrate the efficacy of CLEAR for gaining a better understanding of the inner workings of DNNs during the decision-making process.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, neutral): “…Yeung, and Hauptmann, 2015; Goyal, et al., 2016a,b,c; Hendricks et al., 2016; Hinton, et al., 2015; Jain et al., 2016; Krizhevsky, et al., 2015; Kumar, Wong, and Taylor, 2017; Landecker, Thomure, Bettencourt, Mitchell, Kenyon, and Brumby, 2013; Lei, Barzilay, and Jaakkola, 2016; Li, Chen,…”
Unsupervised Learning of Neural Networks to Explain Neural Networks (2018, background): “Many studies for network interpretability mainly visualized image appearance corresponding to a neural unit inside a CNN [39, 22, 29, 7, 38, 6] or extracted image regions that were responsible for network output [24, 8, 14, 9, 27, 17].”
Interpretable Convolutional Neural Networks (2017, methodology): “[18], influence functions [10] and gradient-based visualization methods [5, 21] and [13] extracted image regions that were responsible for each network output, in order to interpret network representations.”
Interpretability of deep learning models: A survey of results (2017, methodology): “[33] present an alternative heat-mapping method that can show the”
Explainable Artificial Intelligence: a Systematic Review (2020, background): “CLEAR was designed to return the attentive regions responsible for the prediction, along with their attentive levels to understand their influence and the dominant output class associated with these regions.”
Asking 'Why' in AI: Explainability of intelligent systems - perspectives and challenges (2018, background): “Similarly-motivatedapproachesincludetheuseof heatmaps tovisu-alize the relative weighting of parts of an image at the pixel level in terms of a ‘hot to cold’ scale where ‘hottest’ = most highly weighted (Montavon, Lapuschkin, Binder, Samek, and Müller, 2016) programs.”
Opening the Black Box of Financial AI with CLEAR-Trade: A CLass-Enhanced Attentive Response Approach for Explaining and Visualizing Deep Learning-Driven Stock Market Prediction (2017, background): “-driven ﬁnancial AI model. made by the deep neural network only, thus restricting their utility for gaining a more detailed understanding of the decision process. To address this issue, Kumar et. al. [1] recently proposed a CLass Enhanced Attentive Response (CLEAR) approach that not only identiﬁes a”
Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations (2020, background): “, 2016) and class-enhanced attentive response (CLEAR) (Kumar et al., 2017), which allow to determine what specific input features contribute to the decision of a neural network.”