Key challenges for delivering clinical impact with artificial intelligence
By Christopher J. Kelly, Alan Karthikesalingam, Mustafa Suleyman, Greg C. Corrado, Dominic King. 2019
2,399 citations recorded by the source corpus.
BackgroundArtificial intelligence (AI) research in healthcare is accelerating rapidly, with potential applications being demonstrated across various domains of medicine. However, there are currently limited examples of such techniques being successfully deployed into clinical practice. This article explores the main challenges and limitations of AI in healthcare, and considers the steps required to translate these potentially transformative technologies from research to clinical practice.Main bodyKey challenges for the translation of AI systems in healthcare include those intrinsic to the science of machine learning, logistical difficulties in implementation, and consideration of the barriers to adoption as well as of the necessary sociocultural or pathway changes. Robust peer-reviewed clinical evaluation as part of randomised controlled trials should be viewed as the gold standard for evidence generation, but conducting these in practice may not always be appropriate or feasible. Performance metrics should aim to capture real clinical applicability and be understandable to intended users. Regulation that balances the pace of innovation with the potential for harm, alongside thoughtful post-market surveillance, is required to ensure that patients are not exposed to dangerous interventions nor deprived of access to beneficial innovations. Mechanisms to enable direct comparisons of AI systems must be developed, including the use of independent, local and representative test sets. Developers of AI algorithms must be vigilant to potential dangers, including dataset shift, accidental fitting of confounders, unintended discriminatory bias, the challenges of generalisation to new populations, and the unintended negative consequences of new algorithms on health outcomes.ConclusionThe safe and timely translation of AI research into clinically validated and appropriately regulated systems that can benefit everyone is challenging. Robust clinical evaluation, using metrics that are intuitive to clinicians and ideally go beyond measures of technical accuracy to include quality of care and patient outcomes, is essential. Further work is required (1) to identify themes of algorithmic bias and unfairness while developing mitigations to address these, (2) to reduce brittleness and improve generalisability, and (3) to develop methods for improved interpretability of machine learning predictions. If these goals can be achieved, the benefits for patients are likely to be transformational.
Subfields: Artificial Intelligence, Health Informatics, Medical & Health AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Sung Ju Hwang
- James Y. Zou
- Percy Liang
- Yoav Goldberg
- Jianfeng Gao
- Jaehong Yoon
- Marzyeh Ghassemi
In Health Informatics
- Marzyeh Ghassemi
- Kee Yuan Ngiam
- Andrew Beam
- Cuntai Guan
- Erico Tjoa
- Pranav Rajpurkar
- Hock Hai Teo
- Jiamin Yin
In Medical & Health AI
- Marzyeh Ghassemi
- Andrew Beam
- Kee Yuan Ngiam
- Cuntai Guan
- Erico Tjoa
- Ewout W. Steyerberg
- Pranav Rajpurkar
- Gary S. Collins
Citations in the corpus
- A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, background): “demonstrated promising performance in performing medical tasks, they are hardly perfect [1]–[9].”
- Representational Continuity for Unsupervised Continual Learning (2021, background): “We argue that this assumption is not realistic for most of the real-time applications, including self-driving cars (Bojarski et al., 2016), medical applications (Kelly et al., 2019) and conversational agents (Li et al., 2020).”
- COVID-19 Image Data Collection: Prospective Predictions Are the Future (2020, background): “This can lead to many negative outcomes if treated as a typical benchmark dataset [Maguolo and Nanni, 2020; Cohen et al., 2020b; Seyyed-Kalantari et al., 2020; Kelly et al., 2019].”
- Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge (2022, neutral): “Nontumor 967 (19) 1,925 (35) 95 (49) 58 (29) 155 (47) 66 (31) 254 (34) 108 (33) 3,628 (29)”
- Addressing bias in big data and AI for health care: A call for open science (2021, background): “65 Explainable AI models A direction that AI algorithms will need to consider is that of explainable AI.”
- AI in health and medicine (2022, methodology): “To improve generalization, models can undergo site-specific training to adapt to the specific quirks of each place they are deployed, and they can also be trained and validated on datasets collected from different sources 94,96 .”
- Rethinking the Representational Continuity: Towards Unsupervised Continual Learning (2021, background): “We argue that this assumption is not realistic for most of the real-time applications, including self-driving cars (Bojarski et al., 2016), robotics (Li et al., 2020a), medical applications (Kelly et al., 2019) and conversational agents (Li et al., 2020b).”
- Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, background): “In addition, bias in the data is also a major challenge to handle and that can eventually lead to algorithmic bias [178].”
- Artificial intelligence (AI) and big data in cancer and precision oncology (2020, methodology): “To implement AI-based systems for routine clinical practice, the intended users require training and understanding of the system [111].”
- Towards conversational diagnostic artificial intelligence (2025, positive): “Previous research has shown that AI systems may match or exceed human diagnostic performance in specific, narrow tasks 21,22 in retrospective evaluation.”
- Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022, methodology): “…of the AIPM, external AIPM model validation can be based on discrimination (area under the receiver operating characteristic curve, sensitivity, speci fi city, positive and negative predictive values), calibration (calibration plot) 5,6,13,17,49,56,78 , and Decision Curve Analysis 5,6,17,49,68,72 .”