Bias in data‐driven artificial intelligence systems—An introductory survey

By Eirini Ntoutsi, P. Fafalios, U. Gadiraju, Vasileios Iosifidis, Wolfgang Nejdl, Maria-Esther Vidal, Salvatore Ruggieri, Franco Turini, Symeon Papadopoulos, Emmanouil Krasanakis and 13 more. WIREs Data Mining Knowl. Discov., 2020

903 citations recorded by the source corpus.

Artificial Intelligence (AI)‐based systems are widely employed nowadays to make decisions that have far‐reaching impact on individuals and society. Their decisions might affect everyone, everywhere, and anytime, entailing concerns about potential human rights issues. Therefore, it is necessary to move beyond traditional AI algorithms optimized for predictive performance and embed ethical and legal principles in their design, training, and deployment to ensure social good while still benefiting from the huge potential of the AI technology. The goal of this survey is to provide a broad multidisciplinary overview of the area of bias in AI systems, focusing on technical challenges and solutions as well as to suggest new research directions towards approaches well‐grounded in a legal frame. In this survey, we focus on data‐driven AI, as a large part of AI is powered nowadays by (big) data and powerful machine learning algorithms. If otherwise not specified, we use the general term bias to describe problems related to the gathering or processing of data that might result in prejudiced decisions on the bases of demographic features such as race, sex, and so forth.

Subfields: Artificial Intelligence, Health Informatics, Safety Research

View Paper: https://www.semanticscholar.org/paper/17f423a5e542a4bd4de0243548e127038dea6ab5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety Research

Citations in the corpus

Bias and Debias in Recommender System: A Survey and Future Directions (2020, background): “There are some surveys on the bias and fairness of general machine learning and artificial intelligence systems [17], [18], [19].”
Six Human-Centered Artificial Intelligence Grand Challenges (2023, background): “In many cases AI is trained on data derived from human behaviors and thus may adopt inclinations or behaviors that result from implicit bias in that data (Ntoutsi et al., 2020).”
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle (2019, background): “[37] distinguish issues caused by data generation, data collection, or institutional bias; and Mehrabi et al.”
Deep Neural Networks and Tabular Data: A Survey (2021, background): “Finally, model explanations can also be used to identify and mitigate potential bias or eliminate unfair discrimination against certain groups [208].”
Artificial intelligence in the creative industries: a review (2020, background): “A review of various sources of bias in machine learning is provided in [337].”
A survey on datasets for fairness‐aware machine learning (2021, background): “However, the general policies on data quality or collection still need to be studied and proposed (Ntoutsi et al., 2020).”
Generative artificial intelligence (2023, background): “Factors such as non-representative, imbalanced sampling, incorrect labeling, and mismeasured features during the selection and processing of datasets hinder an unbiased training of the GAI model, ultimately leading to biased algorithmic outcomes (Mehrabi et al., 2022; Ntoutsi et al., 2020).”
The Impact of Artificial Intelligence on Workers' Skills: Upskilling and Reskilling in Organisations (2023, background): “In general, artificial intelligence is not always exempt from bias, such as race -b ased or sex - based (e.g., Ntoutsi et al., 2020).”