Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study.

By Hyo-Eun Kim, Hak Hee Kim, B. Han, Ki Hwan Kim, Kyunghwa Han, Hyeonseob Nam, E. Lee, Eun-Kyung Kim. 2020

404 citations recorded by the source corpus.

BACKGROUND Mammography is the current standard for breast cancer screening. This study aimed to develop an artificial intelligence (AI) algorithm for diagnosis of breast cancer in mammography, and explore whether it could benefit radiologists by improving accuracy of diagnosis. METHODS In this retrospective study, an AI algorithm was developed and validated with 170 230 mammography examinations collected from five institutions in South Korea, the USA, and the UK, including 36 468 cancer positive confirmed by biopsy, 59 544 benign confirmed by biopsy (8827 mammograms) or follow-up imaging (50 717 mammograms), and 74 218 normal. For the multicentre, observer-blinded, reader study, 320 mammograms (160 cancer positive, 64 benign, 96 normal) were independently obtained from two institutions. 14 radiologists participated as readers and assessed each mammogram in terms of likelihood of malignancy (LOM), location of malignancy, and necessity to recall the patient, first without and then with assistance of the AI algorithm. The performance of AI and radiologists was evaluated in terms of LOM-based area under the receiver operating characteristic curve (AUROC) and recall-based sensitivity and specificity. FINDINGS The AI standalone performance was AUROC 0·959 (95% CI 0·952-0·966) overall, and 0·970 (0·963-0·978) in the South Korea dataset, 0·953 (0·938-0·968) in the USA dataset, and 0·938 (0·918-0·958) in the UK dataset. In the reader study, the performance level of AI was 0·940 (0·915-0·965), significantly higher than that of the radiologists without AI assistance (0·810, 95% CI 0·770-0·850; p<0·0001). With the assistance of AI, radiologists' performance was improved to 0·881 (0·850-0·911; p<0·0001). AI was more sensitive to detect cancers with mass (53 [90%] vs 46 [78%] of 59 cancers detected; p=0·044) or distortion or asymmetry (18 [90%] vs ten [50%] of 20 cancers detected; p=0·023) than radiologists. AI was better in detection of T1 cancers (73 [91%] vs 59 [74%] of 80; p=0·0039) or node-negative cancers (104 [87%] vs 88 [74%] of 119; p=0·0025) than radiologists. INTERPRETATION The AI algorithm developed with large-scale mammography data showed better diagnostic performance in breast cancer detection compared with radiologists. The significant improvement in radiologists' performance when aided by AI supports application of AI to mammograms as a diagnostic support tool. FUNDING Lunit.

Subfields: Artificial Intelligence, Oncology, Pulmonary and Respiratory Medicine

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Pulmonary and Respiratory Medicine

Pranav Rajpurkar

Citations in the corpus

AI in health and medicine (2022, background): “For instance, AI assistance has some-times been shown to improve clinical experts’ sensitivity while lowering their specificity 8,86 , and some studies, both prospective and retrospective, have found that combined AI–human performance could not surpass the performance of AI alone 90,91 .”
From patterns to patients: Advances in clinical machine learning for cancer diagnosis, prognosis, and treatment. (2023, background): “designed a reader study in which radiologists evaluated mammograms either with or without the aid of anMLmodel trained onmammograms from five institutions in three countries.(34) Radiologists from multiple institutions had superior performance when working in conjunction with ML rather than alone.”
Revolutionizing healthcare: the role of artificial intelligence in clinical practice (2023, background): “Also, AI was better at detecting early breast cancer (91%) than radiologists 74% [12].”