Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine.

By Peter Lee, Sébastien Bubeck, J. Petro. New England Journal of Medicine, 2023

1,309 citations recorded by the source corpus.

Subfields: Health Informatics, Health Information Management, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/72b74bcff8fd76eff6789111a7ce5d0d6c5ac4db

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health Information Management

1. Tristan Naumann

In Radiology, Nuclear Medicine and Imaging

1. Jianwei Yang
2. Jianfeng Gao
3. Mark Yatskar
4. Pang Wei Koh

Citations in the corpus

• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day (2023, background): “Consequently, although conversational generative AI has demonstrated great potential for biomedical applications [20, 33, 19], current investigations are often limited to unimodal text.”
• The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review (2024, background): “Lee et al. (2023) argue that in medical contexts, this hallucination can be especially risky because they might be nuanced, and the chatbot often delivers them in a persuasive way that can lead the user to believe its accuracy.”
• A study of generative large language model for medical research and healthcare (2023, neutral): “Generative large language models (LLMs) such as the ChatGPT 1 have surprised the world by answering questions conversationally and generating textual content such as emails, articles, and even computer codes, triggering enormous enthusiasm in applying LLMs to healthcare 2 – 4 .”
• Causal Reasoning and Large Language Models: Opening a New Frontier for Causality (2023, background): “Impres-sively, when asked to explain their outputs, update their conclusions given new evidence, and even generate coun-terfactuals, LLMs can create plausible responses (Nori et al., 2023; Lee et al., 2023a,b).”
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023, result): “This is in contrast to prior work that claims LMs approach human performance, even for complex tasks (Ding et al., 2022; Nori et al., 2023; Lee et al., 2023) even though the task of writing biographies is fairly easy.”
• Large language models in medicine (2023, background): “This has led to suggestions that AI is ready to replace doctors, but the reality is not quite so dramatic 17,40–42 .”
• Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs (2024, neutral): “Large language models (LLMs) have shown good performance in various natural language processing (NLP) tasks, such as summarizing, translating, code synthesis, and even logical reasoning 1 – 3 .”
• ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge (2023, background): “Nevertheless, owing to the complex nature of medical information [16], any concealed inaccuracies in diagnoses and health advice could lead to severe outcomes [17].”
• MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data (2023, neutral): “Provided the model has been trained on a sufficient number of medical documents, it may possess the medical knowledge necessary to assist in consultations by supplying accurate information derived from its base texts [4].”
• ExpertQA: Expert-Curated Questions and Attributed Answers (2023, background): “For example, medical professionals are using these systems for performing differential diagnosis (Lee et al., 2023) and researchers are using them for faster literature surveys (Krenn et al., 2022; Birhane et al., 2023; Owens, 2023).”
• ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health (2023, background): “(18) GPT-4 impact on health care delivery and medical research is expected to be huge, but its limitations need to be taken into account (19).”
• HealthBench: Evaluating Large Language Models Towards Improved Human Health (2025, neutral): “…scaling language models, there has been increased interest in their application in healthcare (Clusmann et al., 2023; Thirunavukarasu et al., 2023; Lee et al., 2023; Liu et al., 2023; Moor et al., 2023; Singhal et al., 2023; Omiye et al., 2024; Zakka et al., 2024; Nori et al., 2023; Cosentino et…”
• BioRAG: A RAG-LLM Framework for Biological Question Reasoning (2024, background): “However, this ever-changing landscape and the complex interplay between different knowledge components present obstacles (Lee et al., 2023; Castro Nascimento and Pimentel, 2023; Lecler et al., 2023; Song et al., 2020) in creating efficient domain-specific question-reasoning systems.”