The imperative for regulatory oversight of large language models (or generative AI) in healthcare

By Bertalan Meskó, Eric J. Topol. npj Digit. Medicine, 2023

862 citations recorded by the source corpus.

The rapid advancements in artificial intelligence (AI) have led to the development of sophisticated large language models (LLMs) such as GPT-4 and Bard. The potential implementation of LLMs in healthcare settings has already garnered considerable attention because of their diverse applications that include facilitating clinical documentation, obtaining insurance pre-authorization, summarizing research papers, or working as a chatbot to answer questions for patients about their specific data and concerns. While offering transformative potential, LLMs warrant a very cautious approach since these models are trained differently from AI-based medical technologies that are regulated already, especially within the critical context of caring for patients. The newest version, GPT-4, that was released in March, 2023, brings the potentials of this technology to support multiple medical tasks; and risks from mishandling results it provides to varying reliability to a new level. Besides being an advanced LLM, it will be able to read texts on images and analyze the context of those images. The regulation of GPT-4 and generative AI in medicine and healthcare without damaging their exciting and transformative potential is a timely and critical challenge to ensure safety, maintain ethical standards, and protect patient privacy. We argue that regulatory oversight should assure medical professionals and patients can use LLMs without causing harm or compromising their data or privacy. This paper summarizes our practical recommendations for what we can expect from regulators to bring this vision to reality.

Subfields: Health Informatics, Public Health, Environmental and Occupational Health, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/0893549771094fac547432cb4f84e9605c911a86

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. S. Samsi
2. Dan Zhao
3. Joseph McDonald
4. Baolin Li
5. Adam Michaleas
6. Michael Jones
7. W. Bergeron
8. Devesh Tiwari

Citations in the corpus

• A Comprehensive Overview of Large Language Models (2023, neutral): “These developments underscore the imperative for regulatory oversight to manage the ethical and social challenges accompanying LLMs’ widespread use [427].”
• The TRIPOD-LLM reporting guideline for studies using large language models (2025, neutral): “Furthermore, growing partnerships between electronic health record (EHR) vendors, technology companies and healthcare providers have led to deployment horizons that far outpace current regulatory timelines 8,27 .”
• From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference (2023, background): “From ethical concerns ranging from violations of copyright laws [2], [3] to safety concerns arising from the fact that these models are capable of hallucinating or fabricating information, concerns about these models in the educational and medical domain [4], [5], their carbon footprint, and many mo”