Algorithmic Bias in Autonomous Systems

By David Danks, Alex John London. 2017

408 citations recorded by the source corpus.

Algorithms play a key role in the functioning of autonomous systems, and so concerns have periodically been raised about the possibility of algorithmic bias. However, debates in this area have been hampered by different meanings and uses of the term, "bias." It is sometimes used as a purely descriptive term, sometimes as a pejorative term, and such variations can promote confusion and hamper discussions about when and how to respond to algorithmic bias. In this paper, we first provide a taxonomy of different types and sources of algorithmic bias, with a focus on their different impacts on the proper functioning of autonomous systems. We then use this taxonomy to distinguish between algorithmic biases that are neutral or unobjectionable, and those that are problematic in some way and require a response. In some cases, there are technological or algorithmic adjustments that developers can use to compensate for problematic bias. In other cases, however, responses require adjustments by the agent, whether human or autonomous system, who uses the results of the algorithm. There is no "one size fits all" solution to algorithmic bias.

Subfields: Computational Theory and Mathematics, Information Systems, Safety Research, AI Ethics, Fairness & Society, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Information Systems

In Safety Research

In AI Ethics, Fairness & Society

In Robotics & Control

Citations in the corpus

A Survey on Bias and Fairness in Machine Learning (2019, methodology): “The algorithmic design choices, such as use of certain optimization functions, regularizations, choices in applying regression models on the data as a whole or considering subgroups, and the general use of statistically biased estimators in algorithms [44], can all contribute to biased algorithmic d”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “Several recent papers have demonstrated how social goals are, sometimes clumsily, abstracted and formulated to fit into a prediction task [23, 114, 48, 49, 94, 29, 113, 32, 100].”
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “[37, 10] distinguish various concerns that users may have.”
Troubling Trends in Machine Learning Scholarship (2018, background): “ of new suitcase words. In the fairness literature, where legal, philosophical, and statistical language are often overloaded, terms like bias become suitcase words that must be subsequently unpacked [17]. In common speech and as aspirational terms, suitcase words can serve a useful purpose. Perhaps”
On The Stability of Interpretable Models (2018, background): “First, any observational research project must account for variability/bias in data collection [33].”
Algorithms, Platforms, and Ethnic Bias: An Integrative Essay Forthcoming in Phylon: The Clark Atlanta University Review of Race and Culture (2018, positive): “This means developers must carefully consider the effects of the variable(s) in question and whether the benefit outweighs the potential harm from using sensitive information (Danks & London, 2017).”
Attesting Biases and Discrimination using Language Semantics (2019, background): “sounding names (regardless of the existence of arrest records for those names) [30]. The literature in the area of digital discrimination2 very often refers to the related concept of algorithmic bias [5]. Despite playing an important role in digital discrimination, bias does not necessarily lead to ”
Human Trust in Artificial Intelligence: Review of Empirical Research (2020, methodology): “As the environment in which AI functions is usually highly complex and partially random, AI ’ s behavior is not deterministic (Danks & London, 2017), and the complex multilayered process of AI decision-making is generally not transparent.”
A Framework for Understanding Unintended Consequences of Machine Learning (2019, methodology): “Example: geographic diversity in image datasets Ima-geNet is a widely-used image dataset consisting of 1.2 million labeled images (Deng et al., 2009).”
Six Human-Centered Artificial Intelligence Grand Challenges (2023, neutral): “Sources of model biases include several sources (Danks & London, 2017): AI algorithms might be inherently biased to differential use of information to maximize the accuracy rather than focusing on morally relevant and sensitive judgments, which yield the utilization of a statistically biased…”
Generative artificial intelligence (2023, background): “Overfitting is a typical phenomenon that originates from the chosen learning strategies or optimization functions and causes biased algorithmic outcomes (Danks & London, 2017; Hooker, 2021).”
Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development (2020, background): “One of the main reasons for bias in algorithmic decision-making is the quality of input data, because algorithms learn from historical data as an example; thus, the learning process depends on the exposed examples (Friedman and Nissenbaum 1996; Barocas and Selbst 2016; Danks and London 2017).”
Bias in machine learning - what is it good for? (2020, background): “An alternative would be the existing term algorithmic bias [13].”
The ethics of algorithms: key problems and solutions (2020, neutral): “In this way, one type of problematic algorithmic bias is counterbalanced by another type of algorithmic bias or by introducing compensatory bias when interpreting algorithmic outputs (Danks and London 2017).”
Undesirable biases in NLP: Averting a crisis of measurement (2022, background): “…downstream harms and stereotypes (Blodgett et al., 2020; Dev et al., 2022), others propose a statistical definition that measures bias as the degree to which system behaviors deviate when treating different social groups of interest (Danks & London, 2017; Barocas et al., 2019; Shah et al., 2020).”