A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
By Erico Tjoa, Cuntai Guan. 2019
2,119 citations recorded by the source corpus.
Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning (DL). Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example, the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the DL is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide “obviously” interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that: 1) clinicians and practitioners can subsequently approach these methods with caution; 2) insight into interpretability will be born with more considerations for medical practices; and 3) initiatives to push forward data-based, mathematically grounded, and technically grounded medical education are encouraged.
Subfields: Artificial Intelligence, Health Informatics, Safety, Robustness & Interpretability, Medical Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ana Marasovic
- Alon Jacovi
- Enrique Herrera-Viedma
- Yoav Goldberg
- Dino Pedreschi
- Riccardo Guidotti
- Fosca Giannotti
- Francisco Herrera
In Health Informatics
- Timo Speith
- Ala Al-Fuqaha
- Tim Miller
- Gesina Schwalbe
- Ahmed Bouridane
- Francisco Herrera
- Dejing Dou
- Junaid Qadir
In Safety, Robustness & Interpretability
- Ana Marasovic
- Riccardo Guidotti
- Dino Pedreschi
- Yoav Goldberg
- Alon Jacovi
- Fosca Giannotti
- Timo Speith
- Christian Walter Peter Omlin
In Medical Imaging
Citations in the corpus
- Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “ INTRODUCTION With the rise of opaque and poorly-understood machine learning models in the field of AI, trust is often cited as a key desirable property of the interaction between any user and the AI [8, 11, 71, 76]. The recent rapid growth in the field ofexplainable AI (XAI) is also, in part, motiv”
- The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies (2020, background): “Different taxonomies have been proposed based on the explanation-generating mechanism, the type of explanator, the scope of explanation, the type of model it can explain, or a combination of these features [14, 35].”
- Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities (2021, background): “Deployment Improving explanations with ontologies [18] XAI and privacy [35, 58] XAI and security [2, 64, 62, 30] XAI and safety [65, 66, 2] Human-machine teaming [40, 30, 28, 66, 1, 50, 59, 24], [45, 46]* Explainable agency [67] Machine-to-machine explanation [29, 68, 1] XAI and reinforcement learni”
- Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation (2023, background): “In a technical sense, trustworthiness is the confidence of whether a system/model will act as intended when facing a given problem [16].”
- Deep Neural Networks and Tabular Data: A Survey (2021, methodology): “With stricter regulations for automated decision-making [41] and the adoption of machine learning models in high-stakes domains such as finance and healthcare [45], [153], [154], interpretability is becoming a key concern.”
- A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods (2022, background): “[55] construct a taxonomy solely for the domain of computational pathology, and Tjoa and Guan [70] construct one for the medical domain.”
- Counterfactual Explanations for Machine Learning: A Review (2020, background): “Explainable machine learning has also seen interest from other communities, specifically healthcare [103], having huge social implications.”
- Benchmarking and survey of explanation methods for black box models (2021, background): “…literature reviews related to explainability are focused not just on ML and AI but also on social studies (Miller 2019; Byrne 2019), recommendation systems (Zhang and Chen 2020), model-agents(Anjomshoae et al. 2019), and domain-specific applications such as health and medicine (Tjoa and Guan 2019).”
- Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, background): “[13] 2019 √ Categorization of XAI methods and partially discussed application for healthcare.”
- Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review (2020, neutral): “Explainable machine learning has also seen interest from other communities, specifically healthcare [300], having huge social im-”
- Post-hoc Interpretability for Neural NLP: A Survey (2021, neutral): “In contrast to other surveys and tutorials on interpretability methods [14, 15, 17, 25, 35, 79, 113, 119, 125] which only discusses the most popular approaches (usually 3 among input features, adversarial examples, inluential examples, projection, and linguistic information), this survey ofers a mor”
- Combinatorial Methods for Explainable AI (2020, background): “Full development of explainable AI will require extensive validation through human testing, which has not been included in most work in the field [9].”
- Artificial intelligence in histopathology: enhancing cancer research and clinical oncology (2022, background): “These visualizations and newer meth-ods for explainability such as activation atlases 136 or identification of specific multimodal neurons 137 allow human experts to partly recapitulate the algorithm’s decision by offering some level of perceptive and mathematical interpretability 138 .”
- Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “Interpretation algorithms proposed in this specific domain have been surveyed [154, 163].”
- FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2023, neutral): “60 Biomedical and health data can be subject to major variations in the real world (both expected and unexpected), which can affect the performance of AI tools.”