Mitigating bias in algorithmic hiring: evaluating claims and practices

By Manish Raghavan, Solon Barocas, Jon M. Kleinberg, Karen E. C. Levy. 2019

826 citations recorded by the source corpus.

There has been rapidly growing interest in the use of algorithms in hiring, especially as a means to address or mitigate bias. Yet, to date, little is known about how these methods are used in practice. How are algorithmic assessments built, validated, and examined for bias? In this work, we document and analyze the claims and practices of companies offering algorithms for employment assessment. In particular, we identify vendors of algorithmic pre-employment assessments (i.e., algorithms to screen candidates), document what they have disclosed about their development and validation procedures, and evaluate their practices, focusing particularly on efforts to detect and mitigate bias. Our analysis considers both technical and legal perspectives. Technically, we consider the various choices vendors make regarding data collection and prediction targets, and explore the risks and trade-offs that these choices pose. We also discuss how algorithmic de-biasing techniques interface with, and create challenges for, antidiscrimination law.

Subfields: Safety Research, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Sociology and Political Science

Citations in the corpus

Characterizing Fairness Over the Set of Good Models Under Selective Labels (2021, background): “Algorithmic risk assessments are used to inform decisions in high-stakes settings such as health care, child welfare, criminal justice, consumer lending and hiring (Caruana et al., 2015; Chouldechova et al., 2018; Kleinberg et al., 2018; Fuster et al., 2020; Raghavan et al., 2020).”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, background): “[65] report that many of the hiring companies whose statements they analyzed seem to rely on this approach.”
Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development (2020, methodology): “If the companies use algorithms provided by an external service provider, the algorithms’ code and training data are not transparent for the companies (Raghavan et al. 2020; Sánchez-Monedero et al. 2020).”
Gendered Language in Resumes and its Implications for Algorithmic Bias in Hiring (2022, background): “resume screening) can be biased, and how that can lead to discrimination in hiring (Raghavan et al., 2020; Chen et al., 2018; Sühr et al., 2021; Peng et al., 2019).”
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle (2019, background): “For example, algorithms that determine whether someone is offered a loan or a job [12, 36] risk inflicting allocative harm.”
Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases (2020, methodology): “These algorithms are used in important real-world settings, from applicant video screening [36, 61] to autonomous vehicles [29, 52], despite the fact that their harmful downstream effects have been documented in applications such as online ad delivery [67] and image captioning [39].”
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models (2023, background): “This could lead to the perpetuation of existing inequalities in the job market, limiting opportunities for aﬀected individuals and reducing diversity in the workforce [108, 109].”
Participatory provenance as representational auditing for AI-mediated public consultation (2026, unknown): “The literature on AI bias has developed sophisticated frameworks for auditing classification errors in high-stakes decision systems such as hiring [Raghavan et al., 2020], lending [Bartlett et al., 2022], criminal justice [Chouldechova, 2017, Corbett-Davies et al., 2017] and facial recognition…”