Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers

By Catherine A. Gao, Frederick M. Howard, Nikolay S. Markov, Emma C. Dyer, Siddhi Ramesh, Yuan Luo, Alexander T. Pearson. npj Digit. Medicine, 2023

593 citations recorded by the source corpus.

Large language models such as ChatGPT can produce increasingly realistic text, with unknown information on the accuracy and integrity of using these models in scientific writing. We gathered fifth research abstracts from five high-impact factor medical journals and asked ChatGPT to generate research abstracts based on their titles and journals. Most generated abstracts were detected using an AI output detector, ‘GPT-2 Output Detector’, with % ‘fake’ scores (higher meaning more likely to be generated) of median [interquartile range] of 99.98% ‘fake’ [12.73%, 99.98%] compared with median 0.02% [IQR 0.02%, 0.09%] for the original abstracts. The AUROC of the AI output detector was 0.94. Generated abstracts scored lower than original abstracts when run through a plagiarism detector website and iThenticate (higher scores meaning more matching text found). When given a mixture of original and general abstracts, blinded human reviewers correctly identified 68% of generated abstracts as being generated by ChatGPT, but incorrectly identified 14% of original abstracts as being generated. Reviewers indicated that it was surprisingly difficult to differentiate between the two, though abstracts they suspected were generated were vaguer and more formulaic. ChatGPT writes believable scientific abstracts, though with completely generated data. Depending on publisher-specific guidelines, AI output detectors may serve as an editorial tool to help maintain scientific standards. The boundaries of ethical and acceptable use of large language models to help scientific writing are still being discussed, and different journals and conferences are adopting varying policies.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/7d648061157109e8a1a8b382b15ded7ffe690c25

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, background): “In one report, humans detected 68% of generated abstracts, but also flagged 14% of human abstracts as machine generated [130].”
The Potential of Generative Artificial Intelligence Across Disciplines: Perspectives and Future Directions (2023, unknown): “For example, if the output of generative AI tools goes unnoticed by standard plagiarism detection systems used by universities, 79 how can it be confirmed that a particular paper or essay represents the student’s original work rather than just the product of generative AI?”
Evaluating Large Language Models on Medical Evidence Summarization (2023, background): “While recent work has analyzed and evaluated this strategy for news summarization 6 and biomedical literature abstract generation 7 , there is no study yet on medical evidence summarization and appraisal.”
The future landscape of large language models in medicine (2023, background): “For example, ChatGPT is capable of generating scienti ﬁ c abstracts that humans struggle to differentiate from those written by human researchers 54 .”
Understanding generative AI output with embedding models (2025, unknown): “The release of ChatGPT quickly led to noticeable shifts in writing styles ( 60 ), and other studies have issued concerns for the ability of humans to detect articles produced by generative AI models ( 61 – 63 ).”