Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges

By Gabrielle Ras, Marcel A. J. van Gerven, Willem F. G. Haselager. 2018

245 citations recorded by the source corpus.

Issues regarding explainable AI involve four components: users, laws & regulations, explanations and algorithms. Together these components provide a context in which explanation methods can be evaluated regarding their adequacy. The goal of this chapter is to bridge the gap between expert users and lay users. Different kinds of users are identified and their concerns revealed, relevant statements from the General Data Protection Regulation are analyzed in the context of Deep Neural Networks (DNNs), a taxonomy for the classification of existing explanation methods is introduced, and finally, the various classes of explanation methods are analyzed to verify if user concerns are justified. Overall, it is clear that (visual) explanations can be given about various aspects of the influence of the input on the output. However, it is noted that explanation methods or interfaces for lay users are missing and we speculate which criteria these methods / interfaces should satisfy. Finally it is noted that two important concerns are difficult to address with explanation methods: the concern about bias in datasets that leads to biased DNNs, as well as the suspicion about unfair outcomes.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “[74] G. Ras, M. van Gerven, and P. Haselager.”
Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings (2018, background): “A number of such models have been shown to make decisions based on the gender or ancestral origin of an individual, leading to concerns about their “fairness” [1, 18, 22].”
Expanding Explainability: Towards Social Transparency in AI systems (2021, background): “Although there is no established consensus on the complete set of factors that makes an AI system explainable, XAI work commonly shares the goal of making an AI system’s functioning or decisions easy to understand by people [7, 14, 27, 31, 34, 62, 70, 82].”
Human-Centered AI (2021, background): “At present, researchers are exploring the work of oversight and related issues, including regulatory policy and data ownership, through subfields such as responsible AI [20], [21] , algorithmic fairness [22] , ethical AI [23] , and explainable AI [24] .”
Teach Me to Explain: A Review of Datasets for Explainable NLP (2021, methodology): “…et al., 2018; Miller, 2019; Clin-ciu and Hastie, 2019; Barredo Arrieta et al., 2020; Jacovi and Goldberg, 2020), methods (Biran and Cotton, 2017; Ras et al., 2018; Guidotti et al., 2019), evaluation (Hoffman et al., 2018; Yang et al., 2019), or a combination of these topics (Lipton, 2018;…”
Questioning the AI: Informing Design Practices for Explainable AI User Experiences (2020, background): “There have been many efforts to create taxonomies of XAI methods [5, 9, 19, 34, 40, 64, 65, 73, 79].”
Explainable Artificial Intelligence: Objectives, Stakeholders, and Future Research Opportunities (2020, methodology): “Generalizability refers to the range of models which the XAI technique can explain or be applied to, whereby a high generalizability increases the usefulness of the explanation technique (Ras et al., 2018).”
Explainable Sentiment Analysis with Applications in Medicine (2018, methodology): “The taxonomy presented in [45] grouped main approaches into three categories: rule-extraction, attribution and intrinsic methods.”
Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation (2019, unknown): “In the realm of local interpretability, attribution methods have received particular attention in the last years (Ras et al., 2018).”
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies (2020, background): “Definition 2: interpretability An explanation is interpretable if [28]4:”
Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems (2018, background): “These were identified through discussion of many different scenarios involving machine learning systems (a few of which we outline in the next section), and build on ideas from the literature (Weller, 2017; Doshi-Velez & Kim, 2017; Miller, 2017; Ras et al., 2018).”
Evaluating Explanations by Cognitive Value (2018, unknown): “There are at least four distinct kinds of users [3, 4].”
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence (2019, background): “1 Although this research program has already commanded significant attention (see, e.g., Burrell 2016; Doran et al. 2017; Lipton 2016; Ras et al. 2018; Zerilli et al. 2018), conceptual questions remain about the dual notions of “ opacity ” and “ transparency ” and, thus, about what it actually…”