Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management

By Min Kyung Lee. 2018

1,082 citations recorded by the source corpus.

Algorithms increasingly make managerial decisions that people used to make. Perceptions of algorithms, regardless of the algorithms' actual performance, can significantly influence their adoption, yet we do not fully understand how people perceive decisions made by algorithms as compared with decisions made by humans. To explore perceptions of algorithmic management, we conducted an online experiment using four managerial decisions that required either mechanical or human skills. We manipulated the decision-maker (algorithmic or human), and measured perceived fairness, trust, and emotional response. With the mechanical tasks, algorithmic and human-made decisions were perceived as equally fair and trustworthy and evoked similar emotions; however, human managers' fairness and trustworthiness were attributed to the manager's authority, whereas algorithms' fairness and trustworthiness were attributed to their perceived efficiency and objectivity. Human decisions evoked some positive emotion due to the possibility of social recognition, whereas algorithmic decisions generated a more mixed response – algorithms were seen as helpful tools but also possible tracking mechanisms. With the human tasks, algorithmic decisions were perceived as less fair and trustworthy and evoked more negative emotion than human decisions. Algorithms' perceived lack of intuition and subjective judgment capabilities contributed to the lower fairness and trustworthiness judgments. Positive emotion from human decisions was attributed to social recognition, while negative emotion from algorithmic decisions was attributed to the dehumanizing experience of being evaluated by machines. This work reveals people's lay concepts of algorithmic versus human decisions in a management context and suggests that task characteristics matter in understanding people's experiences with algorithmic technologies.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Safety Research

Citations in the corpus

How Do Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness (2018, background): “Since the public is affected by these algorithmic systems, it is important to investigate public views of algorithmic fairness (Lee and Baykal 2017; Lee, Kim, and Lizarondo 2017; Lee 2018; Binns et al. 2018; Woodruff et al. 2018).”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (2018, background): “Given that “fairness” can be highly context- and applicationspecific [43, 66], there is urgent need for domain-specific edu-”
Algorithmic Bias in Education (2021, background): “Interestingly, Loukina et al. (2019) find that when conducting automated essay scoring among essays written by individuals from six different countries, training nation-specific models actually leads to different skews between groups, increasing algorithmic bias compared to training on all groups to”
Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development (2020, methodology): “Algorithms, instead of humans, make decisions, and this has important individual and societal implications in organizational optimization (Chalfin et al. 2016; Lee 2018; Lindebaum et al. 2019).”
Critical Perspectives on Governance Mechanisms for AI/ML Systems (2020, background): “…statistical fairness (Hardt, Price, & Srebro, 2016) and statistical fairness across pre-deﬁned groups (Verma & Rubin, 2018), while related work has explored human perceptions of algorithmic bias and fairness in everyday contexts (Lee, 2018; Woodruff, Fox, Rousso-Schindler, & Warshaw, 2018).”
FactSheets: Increasing trust in AI services through supplier's declarations of conformity (2019, background): “Although there is no scholarly consensus on the speciﬁc traits that imbue trustworthiness in people or algorithms [1, 2], fairness, explainability, general safety, security, and transparency are some of the issues that have raised public concern about trusting AI and threatened the further adoption…”
Human Trust in Artificial Intelligence: Review of Empirical Research (2020, background): “For instance, Lee (2018) did not find a significant differencein initialtrust betweenAI and human decision-making for analytical tasks.”
Automation, Algorithms, and Beyond: Why Work Design Matters More Than Ever in a Digital World (2020, background): “Algorithmic management, in which algorithms take on management tasks, is also on the rise (Lee, 2018).”
Trust in AI and Its Role in the Acceptance of AI Technologies (2022, background): “Past studies have examined trust in various applications of AI, including algorithmic journalism (Shin, 2020c), healthcare AI (Lee & Rich, 2021), and AI used for hiring and work evaluations (Lee, 2018).”
Artificial intelligence and illusions of understanding in scientific research (2024, neutral): “Positioning AI tools in this way is likely to enhance epistemic trust because perceived objectivity is an important marker of credibility 92–94 .”
A Transformer-based Framework for Neutralizing and Reversing the Political Polarity of News Articles (2021, background): “[97]), it’s still hard to avoid potential bias caused by different writing quality or source credibility [65, 94].”
FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity. (2018, background): “[2] M.”
Human-Centered Approaches to Fair and Responsible AI (2020, background): “An emerging line of work seeks to systematically study human perceptions of algorithmic fairness [17, 24, 14], explain algorithmic decisions to promote trust and a sense of fairness [7, 10, 19], understand human use of algorithmic decisions [13, 12, 23], and develop methods to incorporate them into ”
Algorithmic management in a work context (2021, background): “Taking away these first-hand experiences may risk turning managers into “artificial humans” that are shaped and used by the smart technology, not the other way around (Demetis and Lee, 2018).”
The Challenges of Algorithm-Based HR Decision-Making for Personal Integrity (2019, background): “Furthermore, Lee (2018) found in an experiment that when recruitment decisions and performance evaluations are made by an algorithm they are less likely to be perceived as fair and trustworthy, while simultaneously evoking more negative emotions than human decisions.”