An algorithm for removing sensitive information: Application to race-independent recidivism prediction

By James E. Johndrow, Kristian Lum. 2017

178 citations recorded by the source corpus.

Predictive modeling is increasingly being employed to assist human decision-makers. One purported advantage of replacing or augmenting human judgment with computer models in high stakes settings-- such as sentencing, hiring, policing, college admissions, and parole decisions-- is the perceived "neutrality" of computers. It is argued that because computer models do not hold personal prejudice, the predictions they produce will be equally free from prejudice. There is growing recognition that employing algorithms does not remove the potential for bias, and can even amplify it if the training data were generated by a process that is itself biased. In this paper, we provide a probabilistic notion of algorithmic bias. We propose a method to eliminate bias from predictive models by removing all information regarding protected variables from the data to which the models will ultimately be trained. Unlike previous work in this area, our framework is general enough to accommodate data on any measurement scale. Motivated by models currently in use in the criminal justice system that inform decisions on pre-trial release and parole, we apply our proposed method to a dataset on the criminal histories of individuals at the time of sentencing to produce "race-neutral" predictions of re-arrest. In the process, we demonstrate that a common approach to creating "race-neutral" models-- omitting race as a covariate-- still results in racially disparate predictions. We then demonstrate that the application of our proposed method to these data removes racial disparities from predictions with minimal impact on predictive accuracy.

Subfields: Artificial Intelligence, Safety Research, Sociology and Political Science, Privacy & Security, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Sociology and Political Science

In Privacy & Security

In AI Ethics, Fairness & Society

Citations in the corpus

Lessons from archives: strategies for collecting sociocultural data in machine learning (2019, methodology): “rce (eg. this model was trained on Reddit data thus reflecting users of Reddit). Researchers have covered the consequences of ML trained on datasets lacking diversity and called for better regulation [8, 36]. 9naacl2019.org/program/accepted Lessons from Archives: Strategies for Collecting Sociocultu”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “For example, if arrests are a poor measure of crime, fairness constraints based on arrests may be unappealing [66].”
Fair-by-design explainable models for prediction of recidivism (2019, background): “Some state-of-the-art algorithms may provide unethical practices and inequitable outcomes for minorities [6].”
Wasserstein Fair Classification (2019, methodology): “This approach includes current methods to fairness using Wasserstein distances consisting in achieving SDP through transportation of features (Del Barrio et al., 2019; Johndrow and Lum, 2019).”
The Frontiers of Fairness in Machine Learning (2018, background): “[JL17] build upon this work, introducing a likelihood-based approach that can additionally handle continuous protected attributes, discrete features, and which promotes joint independence between the transformed features and the protected attributes.”
Does mitigating ML's disparate impact require disparate treatment? (2017, background): “[JL17] demonstrate how to construct transformations to ensure that the derived features are jointly independent of Z, and show that this produces distributional parity of the resulting fitted model.”
On Fairness and Calibration (2017, background): “However, it has been argued that Statistical Parity may not be applicable in many scenarios [7, 12, 18, 24], as it attempts to guarantee equal representation.”
Fairness in Criminal Justice Risk Assessments: The State of the Art (2017, background): “The goal is to transform all predictors so that fair prediction can be obtained “while still preserving as much ‘information’ in X as possible (Johndrow and Lum, 2017: 6).”
Provably Fair Representations (2017, background): “Other works have proposed learning a representation function such that using the cleaned data, conditioning on different sensitive variable values yields similar distributions [9, 13, 14].”
Algorithmic Fairness: Choices, Assumptions, and Definitions (2021, background): “Jung et al. (2019) bypass explicit elicitation of m and instead query individuals only on whether |δ(v) − δ(v′)| is small for many pairs of individuals i and i′.”
Fairness in Machine Learning (2020, methodology): “Pre-Processing methods consist in transforming the training data to remove undesired biases [16, 21, 24, 28, 47, 51, 52, 53, 61, 64, 65, 66, 67, 86, 90, 91, 92, 112, 114, 116, 118, 121, 122, 127, 126, 164, 174, 188, 190, 191, 193].”
Obtaining Fairness using Optimal Transport Theory (2018, background): “This approach generalizes to higher dimensions the idea of previous works [FFM + 15] and [JL17], who just considere the unidimensional case, where the transport is written in terms of the distribution funtions.”
Fairness in Machine Learning: A Survey (2020, unknown): “…[52, 98, 202]), mapping individuals to an input space that is independent of specific protected subgroupings (e.g., References [54, 89, 125, 154, 185, 310, 313]), transforming the distribution of model predictions in accordance to specific fairness objectives (e.g., References [153, 292]),…”
Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions (2019, background): “In this regard, the closest work to ours are those of Feldman et al. (2015); Johndrow and Lum (2017), where they also control speciﬁc disparate impact metrics by solving an optimal transport problem.”
Does mitigating ML's impact disparity require treatment disparity? (2017, background): “[17] and [19] also construct representations designed to be marginally independent from Z.”
Noise-tolerant fair classification (2019, methodology): “…the original objective fall into three main categories: — pre-processing methods (Zemel et al., 2013; Louizos et al., 2015; Lum & Johndrow, 2016; Johndrow & Lum, 2017; Calmon et al., 2017; del Barrio et al., 2018; Adler et al., 2018), which usually embed the representation of data into a new…”