Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models

By Emilio Ferrara. First Monday, 2023

394 citations recorded by the source corpus.

As generative language models, exemplified by ChatGPT, continue to advance in their capabilities, the spotlight on biases inherent in these models intensifies. This paper delves into the distinctive challenges and risks associated with biases specifically in large-scale language models. We explore the origins of biases, stemming from factors such as training data, model specifications, algorithmic constraints, product design, and policy decisions. Our examination extends to the ethical implications arising from the unintended consequences of biased model outputs. In addition, we analyze the intricacies of mitigating biases, acknowledging the inevitable persistence of some biases, and consider the consequences of deploying these models across diverse applications, including virtual assistants, content generation, and chatbots. Finally, we provide an overview of current approaches for identifying, quantifying, and mitigating biases in language models, underscoring the need for a collaborative, multidisciplinary effort to craft AI systems that embody equity, transparency, and responsibility. This article aims to catalyze a thoughtful discourse within the AI community, prompting researchers and developers to consider the unique role of biases in the domain of generative language models and the ongoing quest for ethical AI.

Subfields: Artificial Intelligence, Health Informatics, Safety Research

View Paper: https://www.semanticscholar.org/paper/16d83e930a4dab2d49f5d276838ddce79df3f787

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety Research

Citations in the corpus

A survey on large language models for recommendation (2023, neutral): “In conclusion, while we have made substantial strides in understanding and implementing LLMs in recommendation systems, the journey ahead is replete with opportunities for innovation and refinement.”
ChatGPT: Jack of all trades, master of none (2023, background): “Of all that standards are addressing bias and fostering ethical development of AI systems[24].”
The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs) (2024, background): “44 Additionally, it is highlighted that inputting patient data raises ethical questions regarding confidentiality, privacy, and data security.”
Bias and Fairness in Large Language Models: A Survey (2023, background): “, LLM-aided healthcare tools may use proxies associated with demographic factors that exacerbate inequities in patient care (Ferrara, 2023) *African-American English; †Standard American English .”
A Survey on Evaluation of Large Language Models (2023, background): “Different from simply measuring social biases, Ferrara [39] investigated the sources, underlying mechanisms, and corresponding ethical consequences of these biases potentially produced by ChatGPT.”
Generative AI (2023, neutral): “Addressing bias and thus fairness in AI receives increasing attention in the academic literature (Dolata et al. 2022; Schramowski et al. 2022; Ferrara 2023; De-Arteaga et al. 2022; Feuerriegel et al. 2020; von Zahn et al. 2022), but remains an open and ongoing research question.”
Cultural bias and cultural alignment of large language models (2023, methodology): “Our evaluation paradigm can be used as a human-in-the-loop approach to guide improvements in cultural alignment for LLMs [16].”
Generative artificial intelligence (2023, neutral): “Similar to their traditional discriminative AI relatives, GAI models are prone to bias causing biased decisions, disadvantages, and discriminations (Ferrara, 2023; Schramowski et al., 2022).”