Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications

By Wojciech Samek, Grgoire Montavon, Sebastian Lapuschkin, Christopher J. Anders, K. Müller. Proceedings of the IEEE, 2021

1,250 citations recorded by the source corpus.

With the broader and highly successful usage of machine learning (ML) in industry and the sciences, there has been a growing demand for explainable artificial intelligence (XAI). Interpretability and explanation methods for gaining a better understanding of the problem-solving abilities and strategies of nonlinear ML, in particular, deep neural networks, are, therefore, receiving increased attention. In this work, we aim to: 1) provide a timely overview of this active emerging field, with a focus on “post hoc” explanations, and explain its theoretical foundations; 2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations; 3) outline best practice aspects, i.e., how to best include interpretation methods into the standard usage of ML; and 4) demonstrate successful usage of XAI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of ML.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/a14d0102e05b34fb0caec09013f0f9ce0f0fa26d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The role of deep learning in urban water management: A critical review. (2022, background): “Compared to conventional machine learning algorithms, developing explainable deep learning has unique difficulties arising from (Samek et al., 2021): 1) multi-scale and distributed nature of network representations, 2) instability from the high depth of networks, 3) searching a reference data point…”
SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects (2021, background): “Obtaining such an understanding of how ML models, 79 here SpookyNet, solve a prediction problem is crucially important in the sciences as a low test set error 33 alone can not rule out that a model may overﬁt or for example capitalize on various artifacts in data 80 or show “Clever Hans” eﬀects.”
Explainable AI Methods - A Brief Overview (2020, background): “Therefore, LIME is also often referred to as surrogate-based explanation technique [70].”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, background): “Speciﬁcally, the developed AI models and algorithms must be privacy-preserving, adversarial-resilient, robust, fair, optimal and interpretable [95].”
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, background): “A lot of work exists on explaining the decisions of a classifier with attribution methods [44].”
Quo vadis artificial intelligence? (2022, background): “Theoretical guarantee is needed in terms of why it works and what the target is [125–128].”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, methodology): “Samek et al. also provided a brief survey on the post hoc methods for explaining the DL models with theoretical background of each methods [66].”
Label-Free Concept Bottleneck Models (2023, methodology): “Post-hoc explanations (Samek et al., 2021): The approach of post-hoc explanations includes some classic methods such as LIME (Ribeiro et al., 2016) and SHAP (Lundberg & Lee, 2017), which try to explain individual model decisions by identifying which parts of the input data (e.g. pixels) are the…”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “From previous reviews and outlooks for the interpretations [29, 47, 81, 107, 144], we summarize the most important desiderata for the interpretation algorithms, i.”
Higher-Order Explanations of Graph Neural Networks via Relevant Walks (2020, background): “For a more comprehensive set of related works, we refer the reader to the review papers [17], [18] for XAI and [2], [19] for GNNs.”