A Primer on Fairness in Criminal Justice Risk Assessments

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

On Fairness and Calibration (2017, background): “Finally, if both false-positive and false-negative errors incur a negative cost on the individual, we may choose to equalize a weighted combination of the error rates [3, 4, 8], which can be graphically described by the classifiers lying on a convex and negatively-sloped level set (Figure 1d).”
Fairness in Criminal Justice Risk Assessments: The State of the Art (2017, methodology): “That is, ( a + d ) / ( a + b + c + d ) should be the same (Berk, 2016b).”
Measuring Model Biases in the Absence of Ground Truth (2021, background): “One of the most common fairness metrics, Demographic (or Statistical) Parity [3, 11, 15], a quantification of the legal doctrine of Disparate Impact [2], can be applied directly for the given task constraints.”
A Theory of Statistical Inference for Ensuring the Robustness of Scientific Results (2018, background): “Although there does not exist a widely accepted mathematical definition of fairness when assessing risk with machine learning (Berk, 2016), if a machine learning method could reach opposing conclusions about a person or group of persons were small adjustments to a dataset or hyperparameters made, th”