Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

By Arun Das, Paul Rad. arXiv.org, 2020

775 citations recorded by the source corpus.

Nowadays, deep neural networks are widely used in mission critical systems such as healthcare, self-driving vehicles, and military which have direct impact on human lives. However, the black-box nature of deep neural networks challenges its use in mission critical applications, raising ethical and judicial concerns inducing lack of trust. Explainable Artificial Intelligence (XAI) is a field of Artificial Intelligence (AI) that promotes a set of tools, techniques, and algorithms that can generate high-quality interpretable, intuitive, human-understandable explanations of AI decisions. In addition to providing a holistic view of the current XAI landscape in deep learning, this paper provides mathematical summaries of seminal work. We start by proposing a taxonomy and categorizing the XAI techniques based on their scope of explanations, methodology behind the algorithms, and explanation level or usage which helps build trustworthy, interpretable, and self-explanatory deep learning models. We then describe the main principles used in XAI research and present the historical timeline for landmark studies in XAI from 2007 to 2020. After explaining each category of algorithms and approaches in detail, we then evaluate the explanation maps generated by eight XAI algorithms on image data, discuss the limitations of this approach, and provide potential future directions to improve XAI evaluation.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/c483beec0afae8d08f011182460095049025b8d1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2022, methodology): “Since we could not find a complete and recent overview of explanation types, we reviewed recent XAI surveys [16, 59, 171, 287] and adapted and extended the categories identified by Guidotti et al.”
The enlightening role of explainable artificial intelligence in medical & healthcare domains: A systematic literature review (2023, methodology): “LIME and SHAP are the two top most used algorithms in XAI [117].”
Explainable artificial intelligence: an analytical review (2021, background): “This can lead to more reliable systems which are necessary to guarantee safety and meet regulations (Das & Rad, 2020).”
Trust in AI: progress, challenges, and future directions (2024, neutral): “The trustworthiness of these algorithms has been questioned by many ethical, technical, and engineering communities (Das and Rad, 2020; von Eschenbach, 2021).”
Revisiting Methods for Finding Influential Examples (2021, methodology): “Such methods are often referred to as global instance-based interpretability methods (rather than local methods) (Carvalho, Pereira, and Cardoso 2019; Molnar 2019; Das and Rad 2020), and they include TraceIn (Pruthi et al.”
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2023, methodology): “Once trained, the GPT model can generate explanations in natural language for specific black-box models based on their inputs and outputs [22].”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “With the rise of opaque and poorly-understood machine learning models in the field of AI, trust is often cited as a key desirable property of the interaction between any user and the AI [8, 11, 71, 76].”
Human-in-the-loop machine learning: a state of the art (2022, background): “Sometimes these reviews offer perspectives that do not always coincide as occurs with XAI goals which are classified differently in recent review papers (Barredo Arrieta et al. 2020; Minh et al. 2021; Meske et al. 2022; Das and Rad 2020).”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “Some of the earlier milestones for the current field of XAI were methods to explain input importance for models with image inputs (Das and Rad, 2020), such as LIME (Ribeiro et al, 2016) and LRP (Bach et al, 2015).”