Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study

By Sun Huh. Journal of Educational Evaluation for Health Professions, 2023

325 citations recorded by the source corpus.

This study aimed to compare the knowledge and interpretation ability of ChatGPT, a language model of artificial general intelligence, with those of medical students in Korea by administering a parasitology examination to both ChatGPT and medical students. The examination consisted of 79 items and was administered to ChatGPT on January 1, 2023. The examination results were analyzed in terms of ChatGPT’s overall performance score, its correct answer rate by the items’ knowledge level, and the acceptability of its explanations of the items. ChatGPT’s performance was lower than that of the medical students, and ChatGPT’s correct answer rate was not related to the items’ knowledge level. However, there was a relationship between acceptable explanations and correct answers. In conclusion, ChatGPT’s knowledge and interpretation ability for this parasitology examination were not yet comparable to those of medical students in Korea.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/3c76d5b253a3f8f2bb2294e94037b6fe16bf1ab1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023, background): “Huh [56] A study to compare ChatGPT performance on a parasitology exam to the performance of Korean medical students ChatGPT performance will improve by deep learning ChatGPT performance was lower compared to medical students; plausible explanations for incorrect answers (hallucination) ChatGPT perf”
ChatGPT in Healthcare: A Taxonomy and Systematic Review (2023, background): “[51] medical education compare ChatGPT with medical students in (an internal) parasitology exam (79 questions) ChatGPT is not comparable to medical student (Acc.”
Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions (2023, background): “LLMs have the potential to significantly impact all phases of medical education programs, offering numerous benefits in various aspects, including curriculum planning, delivery, assessments, programmatic enhancements, and research [8-30].”
Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023, background): “Despite the remarkable performance of these models on standardized tests, they still commit significant errors, and their performance is often on par with that of novice learners [32,35,36].”