Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers

By Catherine A. Gao, Frederick M. Howard, Nikolay S. Markov, Emma C. Dyer, Siddhi Ramesh, Yuan Luo, Alexander T. Pearson. bioRxiv, 2022

365 citations recorded by the source corpus.

Background Large language models such as ChatGPT can produce increasingly realistic text, with unknown information on the accuracy and integrity of using these models in scientific writing. Methods We gathered ten research abstracts from five high impact factor medical journals (n=50) and asked ChatGPT to generate research abstracts based on their titles and journals. We evaluated the abstracts using an artificial intelligence (AI) output detector, plagiarism detector, and had blinded human reviewers try to distinguish whether abstracts were original or generated. Results All ChatGPT-generated abstracts were written clearly but only 8% correctly followed the specific journal’s formatting requirements. Most generated abstracts were detected using the AI output detector, with scores (higher meaning more likely to be generated) of median [interquartile range] of 99.98% [12.73, 99.98] compared with very low probability of AI-generated output in the original abstracts of 0.02% [0.02, 0.09]. The AUROC of the AI output detector was 0.94. Generated abstracts scored very high on originality using the plagiarism detector (100% [100, 100] originality). Generated abstracts had a similar patient cohort size as original abstracts, though the exact numbers were fabricated. When given a mixture of original and general abstracts, blinded human reviewers correctly identified 68% of generated abstracts as being generated by ChatGPT, but incorrectly identified 14% of original abstracts as being generated. Reviewers indicated that it was surprisingly difficult to differentiate between the two, but that the generated abstracts were vaguer and had a formulaic feel to the writing. Conclusion ChatGPT writes believable scientific abstracts, though with completely generated data. These are original without any plagiarism detected but are often identifiable using an AI output detector and skeptical human reviewers. Abstract evaluation for journals and medical conferences must adapt policy and practice to maintain rigorous scientific standards; we suggest inclusion of AI output detectors in the editorial process and clear disclosure if these technologies are used. The boundaries of ethical and acceptable use of large language models to help scientific writing remain to be determined.

Subfields: Artificial Intelligence, Health Informatics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b36acdfc67612d707c95d1ed282672d3ca262be7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Statistics and Probability

Citations in the corpus

Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation (2023, methodology): “Furthermore, researchers have explicated warned against ChatGPT’s potential unethical application (Gao et al., 2022).”
Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review (2023, background): “Instances of technological plagiarism already exist, in which AI has generated abstracts or entire scientific papers with minimal human involvement [66,67].”
The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review (2024, methodology): “Ethical concerns have been raised regarding the potential for AI hallucinations (Gao et al., 2022), algorithmic bias (Alrazaq et al., 2023), plagiarism (De Angelis et al., 2023), privacy issues (Dempere et al., 2023), transparency concerns (Dergaa et al., 2023), and over-reliance (Koos & Wachsmann,…”
Evaluating Large Language Models on Medical Evidence Summarization (2023, neutral): “While recent work has analyzed and evaluated this strategy for news summarization [7] and biomedical literature abstract generation [8], there is no study yet on medical evidence summarization and evaluation.”
Testing of detection tools for AI-generated text (2023, background): “…was able to make an excellent distinction between original and generated abstracts because the majority of the original abstracts were scored extremely low (corresponding to human-written content) while the detector found a high probability of AI-generated text in the majority (33 abstracts)…”
Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study (2024, background): “It has been reported as capable of producing high-quality texts (Gao et al., 2022), offering feedback on text organization, language use and recommending corrections (Ohio University, 2023), logically organizing content, adding appropriate supporting details and conclusion (Fitria, 2023).”
Understanding Generative AI Content with Embedding Models (2024, result): “The release of ChatGPT quickly led to noticeable shifts in writing styles [55], with other studies issuing concern on the human ability to detect articles produced by generative models [56–58].”