Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

By Tiffany H. Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo and 1 more. medRxiv, 2022

3,505 citations recorded by the source corpus.

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, even clinical decision-making.

Subfields: Artificial Intelligence, General Health Professions, Health Informatics

View Paper: https://www.semanticscholar.org/paper/cf1f26e7cbed3958b3c2870656568c299fece6e3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Kang Liu
  2. Zhiyuan Liu
  3. Thien Huu Nguyen
  4. Nikolaos D. Tselikas
  5. Konstantinos I. Roumeliotis
  6. Matei A. Zaharia
  7. Maosong Sun
  8. Shizhu He

In General Health Professions

  1. C. Preiksaitis
  2. Eric Horvitz
  3. Michael Agyemang Adarkwah
  4. Thien Huu Nguyen
  5. Christian Rose
  6. Boulus Shehata
  7. Ronghuai Huang
  8. Ahmed Tlili

In Health Informatics

  1. Nikolaos D. Tselikas
  2. Konstantinos I. Roumeliotis
  3. Matei A. Zaharia
  4. C. Preiksaitis
  5. H. Alkaissi
  6. James Y. Zou
  7. Eric Horvitz
  8. Jan Egger

Citations in the corpus