ChatGPT: The End of Online Exam Integrity?

By Teo Sušnjak. Education sciences, 2022

579 citations recorded by the source corpus.

This study addresses the significant challenge posed by the use of Large Language Models (LLMs) such as ChatGPT on the integrity of online examinations, focusing on how these models can undermine academic honesty by demonstrating their latent and advanced reasoning capabilities. An iterative self-reflective strategy was developed for invoking critical thinking and higher-order reasoning in LLMs when responding to complex multimodal exam questions involving both visual and textual data. The proposed strategy was demonstrated and evaluated on real exam questions by subject experts and the performance of ChatGPT (GPT-4) with vision was estimated on an additional dataset of 600 text descriptions of multimodal exam questions. The results indicate that the proposed self-reflective strategy can invoke latent multi-hop reasoning capabilities within LLMs, effectively steering them towards correct answers by integrating critical thinking from each modality into the final response. Meanwhile, ChatGPT demonstrated considerable proficiency in being able to answer multimodal exam questions across 12 subjects. These findings challenge prior assertions about the limitations of LLMs in multimodal reasoning and emphasise the need for robust online exam security measures such as advanced proctoring systems and more sophisticated multimodal exam questions to mitigate potential academic misconduct enabled by AI technologies.

Subfields: Health Informatics, Safety Research

View Paper: https://www.semanticscholar.org/paper/8822357efe500caded16e603d21239be3a39547c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health Informatics

In Safety Research

Citations in the corpus

ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning (2023, background): “…the third dimension for ChatGPT evaluation discusses the potential impacts and risks of the technology for the broader society, e.g., in education (Susnjak, 2022; Khalil and Er, 2023), law (Choi et al., 2023), medical (Kung et al., 2022), ethnics (Shen et al., 2023), human-computer collaboration…”
Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers (2023, unknown): “Articles are bemoaning the death of the traditional school essay assignment 4,6,7 , as ChatGPT has been shown to generate high-scoring papers 8 , correctly answer USMLE questions 9 , and even articulate critical thinking 10 .”
ChatGPT: Jack of all trades, master of none (2023, background): “[25], [30]) and might be used for cheating in academic tests [25].”
Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (2023, methodology): “Susnjak [79] proposed a solution to the possibility of large language models like ChatGPT being used for exam cheating: guiding ChatGPT to generate some critical thinking problems through questioning, then providing answers and critically evaluating them.”
Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey (2023, background): “In a preprint, [25] assessed ChatGPT’s ability to generate human-like responses to non-trivial university-level questions in various disciplines.”
The opportunities and challenges of ChatGPT in education (2023, unknown): “5 to write essays and other assignments (Pavlik, 2023; Susnjak, 2022).”
Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation (2023, background): “In literature applying ChatGPT in education, similar concerns were heard from participants and scholars (Haque et al., 2022; Susnjak, 2022).”
Zero-Shot Information Extraction via Chatting with ChatGPT (2023, background): “Many ﬁelds received its impacts and evolving fast, such as Medicine (Jeblick et al., 2022; King, 2022) and Online Exam (Susnjak, 2022).”
Transformative Effects of ChatGPT on Modern Education: Emerging Era of AI Chatbots (2023, background): “As identified in the literature [9,13,16], using ChatGPT properly will enable teachers and students to employ more comprehensive teaching and assessment strategies, like project-oriented, inquiry-based, cooperative, personalised, or multidisciplinary education.”
Large Language Models for Education: A survey and outlook (2024, unknown): “In addition, empirical studies have demonstrated the potential of LLMs to function as writing and reading assistants in educational contexts.”
Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT (2023, result): “…on existing educational chatbots, see Kuhail et al., 2022), this exponentially advanced chatbot can generate answers according to the context of a given prompt, and thus, can engage in a conversation that is more akin to human–human interaction than those of previous chatbots (Susnjak, 2022).”
Determinants of Intention to Use ChatGPT for Educational Purposes: Findings from PLS-SEM and fsQCA (2023, unknown): “As a result, students are more engaged and more likely to achieve academic success (Susnjak, 2022).”
ChatGPT is not all you need. A State of the Art Review of large Generative AI models (2023, background): “We believe that it is necessary to provide a state-of-the-art review on the most popular generative AI models as they are revolutionizing several industries like the art industry [2] or universities [16,30].”
Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education (2023, methodology): “ChatGPT has caused significant excitement in the field, as well as some consternation [3] [4].”
On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective (2023, background): “There are reﬂections and discussions from law [10], education [17, 22, 26, 44], human-computer interaction [45], medicine [21], and writing [5].”
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models (2023, methodology): “Susnjak [85] proposed a solution to the possibility of large language models like ChatGPT being used for exam cheating: guiding ChatGPT to generate some critical thinking problems through questioning, then providing answers and critically evaluating them.”
ChatGPT and Open-AI Models: A Preliminary Review (2023, methodology): “Education/Exams Responses given by ChatGPT were on-topic and relevant, achieving high scores on precision, relevance, depth, and originality [15].”