A statistical framework for fair predictive algorithms

By Kristian Lum, James E. Johndrow. 2016

114 citations recorded by the source corpus.

Predictive modeling is increasingly being employed to assist human decision-makers. One purported advantage of replacing human judgment with computer models in high stakes settings-- such as sentencing, hiring, policing, college admissions, and parole decisions-- is the perceived "neutrality" of computers. It is argued that because computer models do not hold personal prejudice, the predictions they produce will be equally free from prejudice. There is growing recognition that employing algorithms does not remove the potential for bias, and can even amplify it, since training data were inevitably generated by a process that is itself biased. In this paper, we provide a probabilistic definition of algorithmic bias. We propose a method to remove bias from predictive models by removing all information regarding protected variables from the permitted training data. Unlike previous work in this area, our framework is general enough to accommodate arbitrary data types, e.g. binary, continuous, etc. Motivated by models currently in use in the criminal justice system that inform decisions on pre-trial release and paroling, we apply our proposed method to a dataset on the criminal histories of individuals at the time of sentencing to produce "race-neutral" predictions of re-arrest. In the process, we demonstrate that the most common approach to creating "race-neutral" models-- omitting race as a covariate-- still results in racially disparate predictions. We then demonstrate that the application of our proposed method to these data removes racial disparities from predictions with minimal impact on predictive accuracy.

Subfields: Artificial Intelligence, Safety Research, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Sociology and Political Science

Citations in the corpus

Empirical Risk Minimization under Fairness Constraints (2018, background): “In recent years there has been a lot of interest on algorithmic fairness in machine learning see, e.g., [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13] and references therein.”
Noise-tolerant fair classification (2019, methodology): “…while optimizing the original objective fall into three main categories: — pre-processing methods (Zemel et al., 2013; Louizos et al., 2015; Lum & Johndrow, 2016; Johndrow & Lum, 2017; Calmon et al., 2017; del Barrio et al., 2018; Adler et al., 2018), which usually embed the…”
Mitigating Unwanted Biases with Adversarial Learning (2018, background): “Simplemodels: Lum and Johndrow [8] demonstrate that removing the protected variable from the training data fails to yield a debiased”
Beyond Parity: Fairness Objectives for Collaborative Filtering (2017, background): “Various studies have considered algorithmic fairness in problems such as supervised classification [20, 23, 28].”
Incomplete Contracting and AI Alignment (2018, background): “The technical literature here (see, e.g. [21, 27, 30, 52, 53]) seeks to develop techniques to align algorithmic decisions with complex human goals such as discriminating between prospective employees on ability but not gender or race.”
The relationship between trust in AI and trustworthy machine learning technologies (2019, background): “• Fairness Technologies: technologies focused on detection or prevention of discrimination and bias in different demographics [98, 63, 77, 97, 55, 23, 49, 37, 46, 25, 50, 96, 35].”
Fairness in Machine Learning: A Survey (2020, background): “, redlining5) [217, 231, 285, 268, 87, 54, 86, 191, 87, 185].”
Fairness in Machine Learning (2020, methodology): “…21, 22, 23, 25, 28, 29, 31, 34, 44, 45, 46, 52, 53, 54, 55, 56, 59, 66, 69, 71, 72, 73, 74, 75, 82, 84, 87, 89, 90, 94, 95, 97, 98, 99, 103, 104, 106, 114, 115, 116, 121, 123, 126, 138, 139, 141, 148, 149, 153, 156, 159, 165, 174, 176, 177, 178, 179, 180, 182, 183, 184, 186, 187, 191, 192, 193].”
New Fairness Metrics for Recommendation that Embrace Differences (2017, background): “Various studies have considered algorithmic fairness in problems such as classification [15, 18, 22].”