Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies

By M. Nagendran, Yang Chen, Christopher A. Lovejoy, A. Gordon, Matthieu Komorowski, H. Harvey, Eric J. Topol, John P. A. Ioannidis, Gary S. Collins, M. Maruthappu. 2020

944 citations recorded by the source corpus.

Abstract Objective To systematically examine the design, reporting standards, risk of bias, and claims of studies comparing the performance of diagnostic deep learning algorithms for medical imaging with that of expert clinicians. Design Systematic review. Data sources Medline, Embase, Cochrane Central Register of Controlled Trials, and the World Health Organization trial registry from 2010 to June 2019. Eligibility criteria for selecting studies Randomised trial registrations and non-randomised studies comparing the performance of a deep learning algorithm in medical imaging with a contemporary group of one or more expert clinicians. Medical imaging has seen a growing interest in deep learning research. The main distinguishing feature of convolutional neural networks (CNNs) in deep learning is that when CNNs are fed with raw data, they develop their own representations needed for pattern recognition. The algorithm learns for itself the features of an image that are important for classification rather than being told by humans which features to use. The selected studies aimed to use medical imaging for predicting absolute risk of existing disease or classification into diagnostic groups (eg, disease or non-disease). For example, raw chest radiographs tagged with a label such as pneumothorax or no pneumothorax and the CNN learning which pixel patterns suggest pneumothorax. Review methods Adherence to reporting standards was assessed by using CONSORT (consolidated standards of reporting trials) for randomised studies and TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) for non-randomised studies. Risk of bias was assessed by using the Cochrane risk of bias tool for randomised studies and PROBAST (prediction model risk of bias assessment tool) for non-randomised studies. Results Only 10 records were found for deep learning randomised clinical trials, two of which have been published (with low risk of bias, except for lack of blinding, and high adherence to reporting standards) and eight are ongoing. Of 81 non-randomised clinical trials identified, only nine were prospective and just six were tested in a real world clinical setting. The median number of experts in the comparator group was only four (interquartile range 2-9). Full access to all datasets and code was severely limited (unavailable in 95% and 93% of studies, respectively). The overall risk of bias was high in 58 of 81 studies and adherence to reporting standards was suboptimal (<50% adherence for 12 of 29 TRIPOD items). 61 of 81 studies stated in their abstract that performance of artificial intelligence was at least comparable to (or better than) that of clinicians. Only 31 of 81 studies (38%) stated that further prospective studies or trials were required. Conclusions Few prospective deep learning studies and randomised trials exist in medical imaging. Most non-randomised trials are not prospective, are at high risk of bias, and deviate from existing reporting standards. Data and code availability are lacking in most studies, and human comparator groups are often small. Future studies should diminish risk of bias, enhance real world clinical relevance, improve reporting and transparency, and appropriately temper conclusions. Study registration PROSPERO CRD42019123605.

Subfields: Artificial Intelligence, Health Informatics, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Ethical Implementation of Artificial Intelligence to Select Embryos in In Vitro Fertilization (2021, background): “The lack of RCTs appears to be typical of much of AI in medicine [23].”
VinDr-CXR: An open dataset of chest X-rays with radiologist’s annotations (2020, background): “These labels were divided into 2 categories: local labels (1-22) and global labels (23-28).”
Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review (2021, result): “7 Similar to our results, major deficiencies were found in the analysis domain including the number of events per variable, inclusion of enrolled participants in the analysis, reporting of relevant model performance measures, and overfitting.”
The myth of generalisability in clinical research and machine learning in health care (2020, neutral): “12 More often, universality is a myth.”
Mapping the Landscape of Artificial Intelligence Applications against COVID-19 (2020, background): “In particular, their performance should be validated on a relevant and diverse set of training, validation, and test datasets, and they should demonstrate effectiveness in the clinical workflow [79] .”
The Fallacy of AI Functionality (2022, result): “Nagendran et al. [129] systematically examined the methods and claims of studies which compared the performance of diagnostic deep learning computer vision algorithms against that of expert clinicians.”
Surgical data science – from concepts toward clinical translation (2020, background): “While long-term clinical studies are a common prerequisite for clinical translation, despite intense research, the number of existing clinical studies in AI-based medicine is extremely low (Nagendran et al., 2020).”
MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care (2020, background): “An empirical evaluation of 81 studies comparing AI models against clinicians showed major problems with lack of transparency, bias, and unjustified claims, likely because key details about the studies were often missing.(12) Given the fast-evolving pace of AI solutions in health care, regulating the”
Trust in AI: progress, challenges, and future directions (2024, neutral): “…of randomized clinical trials to test the performance of AI systems, the lack of transparency of information ﬂ ows within AI applications, the risk of inequity and discrimination introduced by algorithmic biases, and insuf ﬁ cient regulatory clarity (Nagendran et al., 2020; Vollmer et al., 2018).”
Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension (2020, neutral): “Each item was presented for consideration (26 for SPIRIT-AI and 29 for CONSORT-AI).”