Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
By J. Amann, A. Blasimme, Effy Vayena, D. Frey, V. Madai. BMC Medical Informatics and Decision Making, 2020
1,677 citations recorded by the source corpus.
Background Explainability is one of the most heavily debated topics when it comes to the application of artificial intelligence (AI) in healthcare. Even though AI-driven systems have been shown to outperform humans in certain analytical tasks, the lack of explainability continues to spark criticism. Yet, explainability is not a purely technological issue, instead it invokes a host of medical, legal, ethical, and societal questions that require thorough exploration. This paper provides a comprehensive assessment of the role of explainability in medical AI and makes an ethical evaluation of what explainability means for the adoption of AI-driven tools into clinical practice. Methods Taking AI-based clinical decision support systems as a case in point, we adopted a multidisciplinary approach to analyze the relevance of explainability for medical AI from the technological, legal, medical, and patient perspectives. Drawing on the findings of this conceptual analysis, we then conducted an ethical assessment using the “Principles of Biomedical Ethics” by Beauchamp and Childress (autonomy, beneficence, nonmaleficence, and justice) as an analytical framework to determine the need for explainability in medical AI. Results Each of the domains highlights a different set of core considerations and values that are relevant for understanding the role of explainability in clinical practice. From the technological point of view, explainability has to be considered both in terms how it can be achieved and what is beneficial from a development perspective. When looking at the legal perspective we identified informed consent, certification and approval as medical devices, and liability as core touchpoints for explainability. Both the medical and patient perspectives emphasize the importance of considering the interplay between human actors and medical AI. We conclude that omitting explainability in clinical decision support systems poses a threat to core ethical values in medicine and may have detrimental consequences for individual and public health. Conclusions To ensure that medical AI lives up to its promises, there is a need to sensitize developers, healthcare professionals, and legislators to the challenges and limitations of opaque algorithms in medical AI and to foster multidisciplinary collaboration moving forward.
Subfields: Artificial Intelligence, Health Informatics, Safety Research
View Paper: https://www.semanticscholar.org/paper/d7f1a885e32faa2194ccd5f85da4c4fb5d788392
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Longo Luca
- M. de Vos
- Catherine Mooney
- Claudia Mazo
- Brett A. Becker
- Anna Markella Antoniadi
- Alfred Mertins
- Lan Wei
In Health Informatics
In Safety Research
Citations in the corpus
- Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021, methodology): “, what it takes as input, what it does with the environment, and what it produces as output, and (2) explaining the training of the mapping which produces the output by letting it learn from examples, which encompasses unsupervised, supervised, and reinforcement learning” [93] and might suffice for ”
- Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review (2023, background): “In another field, such as medicine, AI’s potential to transform healthcare and medicine significantly is highlighted, indicating its ability to improve both clinical and patient experiences through enhanced data analysis and decision-making [93, 94].”
- Deep learning in cancer diagnosis, prognosis and treatment selection (2021, background): “assay noise or errors in recording) or systematic errors caused by bias within DL models themselves, which can result from the training data not being representative of the population where DL is later applied [27].”
- SleepTransformer: Automatic Sleep Staging With Interpretability and Uncertainty Quantification (2021, background): “Unofficial communications with leading sleep experts point to the scepticism of deep learning models being a black box, which is a common criticism when it comes to the application of artificial intelligence in healthcare and medicine [32].”
- Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, background): “Thus, the explainability of these systems is not only a technological issue but boils down to medical, legal, ethical, and societal questions that need careful consideration [93].”
- FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2023, neutral): “67 It enables end users to interpret the AI model and outputs, understand the capacities and limitations of the AI tool, and intervene when necessary, such as to decide to use it or not.”