'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
By Reuben Binns, M. V. Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, Nigel R. Shadbolt. 2018
779 citations recorded by the source corpus.
Data-driven decision-making consequential to individuals raises important questions of accountability and justice. Indeed, European law provides individuals limited rights to 'meaningful information about the logic' behind significant, autonomous decisions such as loan approvals, insurance quotes, and CV filtering. We undertake three experimental studies examining people's perceptions of justice in algorithmic decision-making under different scenarios and explanation styles. Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions. Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity. Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles---under repeated exposure of one style, scenario effects obscure any explanation effects. Our results suggests there may be no 'best' approach to explaining algorithmic decisions, and that reflection on their automated nature both implicates and mitigates justice dimensions.
Subfields: Artificial Intelligence, Cognitive Neuroscience, Safety Research, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Q. Liao
- Chenhao Tan
- Vivian Lai
- Jennifer Wortman Vaughan
- Mohit Bansal
- Percy Liang
- Noah A. Smith
- Daniel S. Weld
In Cognitive Neuroscience
- Jessica Morley
- Anat Elhalal
- Libby Kinsey
- Sebastian Raisch
- Sebastian Krakowski
- Yiling Chen
- S. Whittaker
- Karen Huang
In Safety Research
In AI Ethics, Fairness & Society
Citations in the corpus
- How Do Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness (2018, background): “…in targeted online advertising, Grgić-Hlača et al. (2018a) and Grgić-Hlača et al. (2018b) studied human perceptions of features used in algorithmic decision making, and Binns et al. (2018) examined people’s perception of justice in algorithmic decision making under different explanation styles.”
- Predictive Multiplicity in Classification (2019, background): “In human-facing applications , such as credit scoring and recidivism prediction, these predictions affect people [7].”
- Actionable Recourse in Linear Classification (2018, background): “Seeing how the lack of autonomy is perceived as a source of injustice in algorithmic decision-making [6, 14, 33], recourse should be considered whenever humans are subject to the predictions of a machine learning model.”
- Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (2018, background): “For example, HCI researchers have empirically studied users’ expectations and perceptions related to fairness in algorithmic systems, finding that these do not always align with existing statistical definitions [16, 71, 72, 108].”
- Efficient Search for Diverse Coherent Explanations (2019, background): “Binns et al. (2018) found evidence that users prefer counterfactual explanations over case-based reasoning.”
- The Principles and Limits of Algorithm-in-the-Loop Decision Making (2019, background): “…making, prior research has explored topics such as what interactions can facilitate the development of machine learning models [9, 29, 47], how to improve human performance with an algorithm’s assistance [12, 37, 48], and the ways in which laypeople perceive algorithmic decisions [4, 28, 52].”
- Artificial Intelligence and Management: The Automation–Augmentation Paradox (2020, background): “Furthermore, humans “ in the loop ” could explain the sys-tem ’ s outputs, rendering algorithmic decisions more accountable and transparent (Binns, Van Kleek, Veale, Lyngs, Zhao, & Shadbolt, 2018).”
- What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research (2021, background): “Regulators call, for instance, for trustworthy systems [5, 6, 10, 11, 29, 96, 115, 131].”
- This Thing Called Fairness (2019, background): “Research by social scientists and humanities scholars has similarly advanced the state of theory and application [5, 11, 80, 136, 149].”
- How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India (2022, background): “, of source-code, datasets, models, and processes surrounding the development of ADS) [21, 30, 55, 64, 83, 105] and explainable decisions (i.”
- Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies (2021, background): “Professional Job promotion [15], meeting scheduling assistance [74], email topic classification [125], cybersecurity monitoring [42], profession prediction [94], military planning (monitor and direct unmanned vehicles) [130].”