Diagnostic Accuracy of Digital Screening Mammography with and without Computer-aided Detection

By C. Lehman, R. Wellman, D. Buist, K. Kerlikowske, A. Tosteson, Diana L. Miglioretti. 2015

619 citations recorded by the source corpus.

Importance After the Food and Drug Administration (FDA) approved computer-aided detection (CAD) for mammography in 1998, and Centers for Medicare and Medicaid Services (CMS) provided increased payment in 2002, CAD technology disseminated rapidly. Despite sparse evidence that CAD improves accuracy of mammographic interpretations, and costs over $400 million dollars a year, CAD is currently used for the majority of screening mammograms in the U.S. Objective To measure performance of digital screening mammography with and without computer-aided detection in U.S. community practice. Design, Setting and Participants We compared the accuracy of digital screening mammography interpreted with (N=495,818) vs. without (N=129,807) computer-aided detection from 2003 through 2009 in 323,973 women. Mammograms were interpreted by 271 radiologists from 66 facilities in the Breast Cancer Surveillance Consortium. Linkage with tumor registries identified 3,159 breast cancers in 323,973 women within one year of the screening. Main Outcomes and Measures Mammography performance (sensitivity, specificity, and screen detected and interval cancers per 1,000 women) was modeled using logistic regression with radiologist-specific random effects to account for correlation among examinations interpreted by the same radiologist, adjusting for patient age, race/ethnicity, time since prior mammogram, exam year, and registry. Conditional logistic regression was used to compare performance among 107 radiologists who interpreted mammograms both with and without computer-aided detection. Results Screening performance was not improved with computer-aided detection on any metric assessed. Mammography sensitivity was 85.3% (95% confidence interval [CI]=83.6–86.9) with and 87.3% (95% CI 84.5–89.7) without computer-aided detection. Specificity was 91.6% (95% CI=91.0–92.2) with and 91.4% (95% CI=90.6–92.0) without computer-aided detection. There was no difference in cancer detection rate (4.1/1000 women screened with and without computer-aided detection). Computer-aided detection did not improve intra-radiologist performance. Sensitivity was significantly decreased for mammograms interpreted with versus without computer-aided detection in the subset of radiologists who interpreted both with and without computer-aided detection (OR 0.53, 95%CI=0.29–0.97). Conclusions and Relevance CAD does not improve diagnostic accuracy of mammography and may result in missed cancers. These results suggest that insurers pay more for computer-aided detection with no established benefit to women.

Subfields: Artificial Intelligence, Oncology, Pulmonary and Respiratory Medicine

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Oncology

In Pulmonary and Respiratory Medicine

Citations in the corpus

Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists. (2019, background): “Unfortunately, no studies to date have found that traditional computer-aided detection systems directly improve screening performance or cost-effectiveness, mainly because of a low specificity (13,14).”
Computer-Aided Diagnosis in the Era of Deep Learning (2020, background): “Unfortunately, Lehman et al. simply concluded that insurers pay more for CAD with no established benefit to women instead of addressing the problems.”
Deep Learning in Medical Imaging: General Overview (2017, background): “Thus, the net benefit gained by using CAD was unclear (3).”
Artificial intelligence in radiology (2018, neutral): “In examining mammograms, some studies have reported that radiologists rarely altered their diagnostic decisions after viewing results from predefined, feature-based CADe systems and that their clinical integration had no statistical significance on the radiologists’ performance 41,42 .”
Assessment of Convolutional Neural Networks for Automated Classification of Chest Radiographs. (2019, methodology): “Further, this method of disease-specific computer-assisted diagnosis may not ultimately be beneficial to the interpreting clinician (16).”
Deep neural network improves fracture detection by clinicians (2018, background): “For the purpose of model development, we retrospectively obtained a collection of radiographs from a specialty hospital in the United States.”
Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening (2019, background): “Multicenter studies have shown that traditional computeraided detection in mammography programs do not improve their diagnostic performance (Lehman et al., 2015).”
Key challenges for delivering clinical impact with artificial intelligence (2019, background): “Excessive warnings and alerts are known to result in alert fatigue [94, 95].”
Fairness of artificial intelligence in healthcare: review and recommendations (2023, background): “Alert fatigue is a manifestation of this bias, as clini‑ cians may ignore important alerts owing to an overwhelm‑ ing number of false alarms [81, 82].”
Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness (2020, background): “82 Moreover, researchers have recently demonstrated that incorrect prompts from mammography based CAD systems can actually decrease the sensitivity of more discerning users by up to 0.145 (95% confidence interval 0.034 to 0.257) for difficult diagnoses.”
Deep Learning to Improve Breast Cancer Detection on Screening Mammography (2017, background): “Unfortunately, data suggests that commercial CAD systems have not led to significant improvement in performance [5–7] and progress has stagnated in the past decade.”