Do as AI say: susceptibility in deployment of clinical decision-aids
By S. Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J. Berkowitz, E. Lermer, Joseph F. Coughlin, John V. Guttag, Errol Colak, Marzyeh Ghassemi. npj Digital Medicine, 2021
431 citations recorded by the source corpus.
Artificial intelligence (AI) models for decision support have been developed for clinical settings such as radiology, but little work evaluates the potential impact of such systems. In this study, physicians received chest X-rays and diagnostic advice, some of which was inaccurate, and were asked to evaluate advice quality and make diagnoses. All advice was generated by human experts, but some was labeled as coming from an AI system. As a group, radiologists rated advice as lower quality when it appeared to come from an AI system; physicians with less task-expertise did not. Diagnostic accuracy was significantly worse when participants received inaccurate advice, regardless of the purported source. This work raises important considerations for how advice, AI and non-AI, should be deployed in clinical environments.
Subfields: Family Practice, Health Informatics, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/43dd4b095a267ed20251a40bceca1e447458016c
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Health Informatics
Citations in the corpus
- Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “When system designers do not take these perceptual differences into consideration it can lead to misinterpretation of output, which is especially problematic in high-risk settings [247, 248].”
- Towards accurate differential diagnosis with large language models (2023, neutral): “Furthermore, observations of standalone diagnostic accuracy often do not guarantee that an AI tool will improve performance in real-world settings as an assistive tool, and it remains unclear how AI and human decision-making should be optimally integrated in medicine 29 .”