Artificial Hallucinations in ChatGPT: Implications in Scientific Writing

By H. Alkaissi, Samy I McFarlane. Cureus, 2023

1,440 citations recorded by the source corpus.

While still in its infancy, ChatGPT (Generative Pretrained Transformer), introduced in November 2022, is bound to hugely impact many industries, including healthcare, medical education, biomedical research, and scientific writing. Implications of ChatGPT, that new chatbot introduced by OpenAI on academic writing, is largely unknown. In response to the Journal of Medical Science (Cureus) Turing Test - call for case reports written with the assistance of ChatGPT, we present two cases one of homocystinuria-associated osteoporosis, and the other is on late-onset Pompe disease (LOPD), a rare metabolic disorder. We tested ChatGPT to write about the pathogenesis of these conditions. We documented the positive, negative, and rather troubling aspects of our newly introduced chatbot’s performance.

Subfields: Genetics, Health Informatics

View Paper: https://www.semanticscholar.org/paper/7be169a70f5db74f40adfd2790021aa3fbf3cb87

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

1. Na Li
2. Jinhee Kim
3. Rita Detrick
4. Seongryeong Yu

In Health Informatics

1. C. Preiksaitis
2. Jan Egger
3. Jianning Li
4. Christian Rose
5. Na Li
6. Amin Dada
7. Jinhee Kim
8. William H. Walters

Citations in the corpus

• Exploring students’ perspectives on Generative AI-assisted academic writing (2024, positive): “Further, GenAI has been widely reported to hal-lucinate content or provide incorrect guidance, which refers to when a GenAI tool generates inaccurate responses that seem realistic (Alkaissi & McFarlane, 2023).”
• Fabrication and errors in the bibliographic citations generated by ChatGPT (2023, background): “These errors are potentially dangerous, and they are exacerbated by the fact that ChatGPT often stands by its incorrect statements when asked to verify them 12,13,18,24,29 .”
• ChatGPT in Healthcare: A Taxonomy and Systematic Review (2023, background): “[5] medical research editorial comment on the hallucination issue of ChatGPT in medical writing Level 1”
• Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions (2023, background): “LLMs have the potential to significantly impact all phases of medical education programs, offering numerous benefits in various aspects, including curriculum planning, delivery, assessments, programmatic enhancements, and research [8-30].”
• Large language models in medicine (2023, background): “‘Watermarking’ protocols could facilitate high-quality text generation with detectable signatures signaling LLM involvement, but this is not currently being implemented in the most popular models 109 .”
• Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023, background): “[28] Confidence expressed by models “Further, AI-generated content can potentially produce misinformation or biased information...” [14] Misinformation propagation “Consequently, the current level of accuracy is not yet sufficient for immediate clinical application in patient care.””