Towards accurate differential diagnosis with large language models
By Daniel McDuff, Mike Schaekermann, Tao Tu, Anil Palepu, Amy Wang, Jake Garrison, Karan Singhal, Yash Sharma, Shekoofeh Azizi, Kavita Kulkarni and 18 more. Nature, 2023
345 citations recorded by the source corpus.
A comprehensive differential diagnosis is a cornerstone of medical care that is often reached through an iterative process of interpretation that combines clinical history, physical examination, investigations and procedures. Interactive interfaces powered by large language models present new opportunities to assist and automate aspects of this process1. Here we introduce the Articulate Medical Intelligence Explorer (AMIE), a large language model that is optimized for diagnostic reasoning, and evaluate its ability to generate a differential diagnosis alone or as an aid to clinicians. Twenty clinicians evaluated 302 challenging, real-world medical cases sourced from published case reports. Each case report was read by two clinicians, who were randomized to one of two assistive conditions: assistance from search engines and standard medical resources; or assistance from AMIE in addition to these tools. All clinicians provided a baseline, unassisted differential diagnosis prior to using the respective assistive tools. AMIE exhibited standalone performance that exceeded that of unassisted clinicians (top-10 accuracy 59.1% versus 33.6%, P = 0.04). Comparing the two assisted study arms, the differential diagnosis quality score was higher for clinicians assisted by AMIE (top-10 accuracy 51.7%) compared with clinicians without its assistance (36.1%; McNemar’s test: 45.7, P < 0.01) and clinicians with search (44.4%; McNemar’s test: 4.75, P = 0.03). Further, clinicians assisted by AMIE arrived at more comprehensive differential lists than those without assistance from AMIE. Our study suggests that AMIE has potential to improve clinicians’ diagnostic reasoning and accuracy in challenging cases, meriting further real-world evaluation for its ability to empower physicians and widen patients’ access to specialist-level expertise. Diagnostic reasoning using an optimized large language model with a dataset comprising real-world medical cases exhibited improved differential diagnostic performance as an assistive tool for clinicians over search engines and standard medical resources.
Subfields: Artificial Intelligence, Health Informatics
View Paper: https://www.semanticscholar.org/paper/76ab9bc1cbf529d1cfe5963f24069539c3199745
Citations in the corpus
- HealthBench: Evaluating Large Language Models Towards Improved Human Health (2025, neutral): “…examined the performance of large language models on a range of targeted tasks, including generating differential diagnoses (Kanjee et al., 2023; McDuff et al., 2025), assisting with clinical documentation (Williams et al., 2024; Hartman et al., 2024; Baker et al., 2024), mental health…”
- Towards conversational diagnostic artificial intelligence (2025, neutral): “Previous research has shown that AI systems may match or exceed human diagnostic performance in specific, narrow tasks 21,22 in retrospective evaluation.”
- Evaluation and mitigation of the limitations of large language models in clinical decision-making (2024, neutral): “LLMs can summarize reports 3–5 , generate reports 2,4 , serve as diagnostic assistants 21,27 and could ultimately autonomously diagnose patients.”
- CNS-Obsidian: A Neurosurgical Vision-Language Model Built From Scientific Publications. (2025, unknown): “Unlike prior studies using simulated scenarios 23,27 , our trial captured the complexities of actual patient care.”