Discrimination in the Age of Algorithms

By Jon M. Kleinberg, Jens Ludwig, Sendhil Mullainathan, Cass R. Sunstein. Social Science Research Network, 2018

382 citations recorded by the source corpus.

The law forbids discrimination. But the ambiguity of human decision-making often makes it extraordinarily hard for the legal system to know whether anyone has actually discriminated. To understand how algorithms affect discrimination, we must therefore also understand how they affect the problem of detecting discrimination. By one measure, algorithms are fundamentally opaque, not just cognitively but even mathematically. Yet for the task of proving discrimination, processes involving algorithms can provide crucial forms of transparency that are otherwise unavailable. These benefits do not happen automatically. But with appropriate requirements in place, the use of algorithms will make it possible to more easily examine and interrogate the entire decision process, thereby making it far easier to know whether discrimination has occurred. By forcing a new level of specificity, the use of algorithms also highlights, and makes transparent, central tradeoffs among competing values. Algorithms are not only a threat to be regulated; with the right safeguards in place, they have the potential to be a positive force for equity.

Subfields: Global and Planetary Change, Safety Research, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/59138f6c6cc957c03532088386cd15079c6d17af

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Global and Planetary Change

In Safety Research

In Sociology and Political Science

Citations in the corpus

Artificial Intelligence, Responsibility Attribution, and a Relational Justification of Explainability (2019, background): “(Note that it is also possible that the human expert is biased and the algorithm does better, as Sunstein 2018 has argued; algorithms can also help to detect discrimination Kleinberg et al. 2019.”
The promise of artificial intelligence: a review of the opportunities and challenges of artificial intelligence in healthcare. (2021, background): “38 For example, racial biases may be introduced in the creation of datasets, with minorities being under-represented thereby leading to lower-than-expected prediction performance.”
Organizational Decision-Making Structures in the Age of Artificial Intelligence (2019, background): “50 Such applications have the potential to turn algorithms into a powerful counterweight to human decision-making errors.”
Integrating explanation and prediction in computational social science (2021, background): “On the other hand, complex predictive models have also been criticized for failing to generalize 19 as well as being uninterpretable 20 and biased 21 .”
Mitigating bias in algorithmic hiring: evaluating claims and practices (2019, methodology): “Prior work has sought to taxonomize the points at which bias can enter machine learning systems, noting that the choice of target variable or outcome to predict, the training data used, and labelling of examples are all potential sources of disparities [6, 59].”
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence (2019, background): “Yet despite some prominent arguments to the contrary (Kleinberg et al. 2019), it is not clear that automated decisions are much more accessible to external scrutiny.”
Algorithmic management in a work context (2021, background): “Focusing primarily on the technology involved in algorithmic management, some perceive algorithms as “technocratic and dispassionate” decision makers that can provide more objective and consistent decisions than humans (Kahneman et al., 2016; Kleinberg et al., 2018).”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “However, machines also provide systematic ways to reduce bias, and some see the mitigation of bias in algorithm decisions as a potential opportunity to move the needle positively (Kleinberg et al., 2018).”
Towards causal benchmarking of bias in face analysis algorithms (2020, methodology): “Human face datasets are particularly scrutinized [2, 20, 43, 45, 46, 54] because methods and models trained on these data can end up being biased along attributes that are protected by the law [44].”