A Categorical Archive of ChatGPT Failures

By Ali Borji. arXiv.org, 2023

493 citations recorded by the source corpus.

Large language models have been demonstrated to be valuable in different fields. ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation by comprehending context and generating appropriate responses. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries, with fluent and comprehensive answers surpassing prior public chatbots in both security and usefulness. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study. Eleven categories of failures, including reasoning, factual errors, math, coding, and bias, are presented and discussed. The risks, limitations, and societal implications of ChatGPT are also highlighted. The goal of this study is to assist researchers and developers in enhancing future language models and chatbots.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/cd0988714ea326642d2b1bb18753e187fec71e42

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (2023, background): “[86] demonstrated ChatGPT’s inconsistency in reasoning, factual errors, mathematics, coding, and bias across eleven related aspects.”
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models (2023, background): “[7] demonstrated ChatGPT’s inconsistency in reasoning, factual errors, mathematics, coding, and bias across eleven related aspects.”
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, methodology): “Although tools like ChatGPT [4] is capable of generating content that usually appears or sounds reasonable, they are often unreliable in terms of factuality [215].”
Tool Learning with Foundation Models (2023, unknown): “, 2022) and GPT-4 (OpenAI, 2023) with human values and preferences, given the discomforting “jailbreak” responses by ChatGPT (Borji, 2023) and New Bing (Roose, 2023), whether these big models will be mild and compliant remains doubtful.”
Large Language Models for Software Engineering: Survey and Open Problems (2023, background): “Borji [75] presented a rigorous, categorised and systematic analysis of LLM code generation failures for ChatGPT.”
Auditing large language models: a three-layered approach (2023, background): “As recent controversies surrounding ChatGPT [74] have shown, LLMs are prone to give biased or incorrect answers to user queries [75].”
Exploring AI Ethics of ChatGPT: A Diagnostic Analysis (2023, background): “Some of the issues are later covered by [65].”
ChatGPT: Jack of all trades, master of none (2023, methodology): “Borji [88] conducted a systematic review of the typical categories of ChatGPT failures.”
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023, background): “A noteworthy and comprehensive record by Borji can be used as a categorical guide for the issues and concerns of ChatGPT use, especially in the context of scientific writing [20].”
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development (2023, background): “It also has the ability to even provide missing details in planning goals autonomously using commonsense knowledge and reasoning [99].”
Testing of detection tools for AI-generated text (2023, background): “This was in many ways a game-changer, as it is capable of creating responses that are hard to distinguish from human-written text (Borji, 2023; Brown et al., 2020).”
Promises and challenges of generative artificial intelligence for human learning (2024, unknown): “For instance, early explo rations have shown that GPT4 can automatically generate instructional materials, such as explanations of programming concepts, examples and quiz questions, thereby boosting learner engagement and satisfac tion 24 .”
Unveiling Security, Privacy, and Ethical Concerns of ChatGPT (2023, background): “[32], compiled a list of ChatGPT’s limitations in different question categories, including reasoning, logic, math,”
Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education (2023, background): “4 Borji categorizes these failures into ten types, including reasoning, factual errors, math, coding, and bias and presents examples of each [22].”
When large language models meet personalization: perspectives of challenges and opportunities (2023, unknown): “One of the concerns is the reliability of models like ChatGPT in terms of factuality, as indicated in the work [268].”