Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey

By Vanessa Buhrmester, David Münch, Michael Arens. Machine Learning and Knowledge Extraction, 2019

459 citations recorded by the source corpus.

Deep Learning is a state-of-the-art technique to make inference on extensive or complex data. As a black box model due to their multilayer nonlinear structure, Deep Neural Networks are often criticized as being non-transparent and their predictions not traceable by humans. Furthermore, the models learn from artificially generated datasets, which often do not reflect reality. By basing decision-making algorithms on Deep Neural Networks, prejudice and unfairness may be promoted unknowingly due to a lack of transparency. Hence, several so-called explanators, or explainers, have been developed. Explainers try to give insight into the inner structure of machine learning black boxes by analyzing the connection between the input and output. In this survey, we present the mechanisms and properties of explaining systems for Deep Neural Networks for Computer Vision tasks. We give a comprehensive overview about the taxonomy of related studies and compare several survey papers that deal with explainability in general. We work out the drawbacks and gaps and summarize further research ideas.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/9a4382bc7aad5b0f33360e7b119929b95eb04f00

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks (2020, background): “Due to a large number of parameters, they are referred to as a ”black-box” [6].”
The alignment problem from a deep learning perspective (2022, background): “First, since present neural networks are effectively black boxes [Buhrmester et al., 2021], we cannot formally verify that they will reliably behave as intended, and need to rely more on informal analysis.”
From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors (2022, background): “Another major drawback for neural networks is the lack of transparency and interpretability [181, 182].”
Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities (2021, background): “It is one of the most raised challenges in the literature of XAI [1, 31, 10, 32, 9, 41, 33, 26, 43, 42, 2, 34, 8, 44, 35, 36, 37, 18, 45, 38, 39, 40, 46].”
Towards Automated Circuit Discovery for Mechanistic Interpretability (2023, background): “…have identified precise high-level predictors of model performance (Kaplan et al., 2020), but transformers are still widely considered ‘black-boxes’ (Alishahi, Chrupała, and Linzen, 2019) like almost all other neural network models (Fong and Vedaldi, 2017; Buhrmester, Münch, and Arens, 2021).”