Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment

By Chelsea Barabas, Madars Virza, Karthik Dinakar, Joichi Ito, Jonathan L. Zittrain. 2017

156 citations recorded by the source corpus.

Actuarial risk assessments might be unduly perceived as a neutral way to counteract implicit bias and increase the fairness of decisions made at almost every juncture of the criminal justice system, from pretrial release to sentencing, parole and probation. In recent times these assessments have come under increased scrutiny, as critics claim that the statistical techniques underlying them might reproduce existing patterns of discrimination and historical biases that are reflected in the data. Much of this debate is centered around competing notions of fairness and predictive accuracy, resting on the contested use of variables that act as "proxies" for characteristics legally protected against discrimination, such as race and gender. We argue that a core ethical debate surrounding the use of regression in risk assessments is not simply one of bias or accuracy. Rather, it's one of purpose. If machine learning is operationalized merely in the service of predicting individual future crime, then it becomes difficult to break cycles of criminalization that are driven by the iatrogenic effects of the criminal justice system itself. We posit that machine learning should not be used for prediction, but rather to surface covariates that are fed into a causal model for understanding the social, structural and psychological drivers of crime. We propose an alternative application of machine learning and causal inference away from predicting risk scores to risk mitigation.

Subfields: Clinical Psychology, Emergency Medical Services, Sociology and Political Science, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Clinical Psychology

In Emergency Medical Services

In Sociology and Political Science

In AI Ethics, Fairness & Society

Citations in the corpus

Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “In particular, causal graphs can be used to develop interventions at earlier points, prior to decision-making [64, 8].”
Fairness and Abstraction in Sociotechnical Systems (2019, background): “An emerging wave of similar critiques has been directed at the field of fair-ML [9, 29, 36].”
Causal Reasoning for Algorithmic Fairness (2018, background): “And, as argued in [1], we can also try to understand the causal implications for the system we are intervening on.”
Towards a critical race methodology in algorithmic fairness (2019, background): “This point dovetails with the critiques levied by abolitionist activists, scholars, and technologists [1, 9, 11].”
Racial categories in machine learning (2018, background): “Reconsidering the problem as one of causal inference and the predicted outcomes of intervention [28, 32], especially in light of the purposes to which prediction and intervention are intended [3] is a promising path forward.”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, background): “Instead, as has been suggested elsewhere [2, 7], algorithmic fairness assessments could be used as a type of diagnostic tool for the whole network of actors and interactions, identifying salient disparities in intake, treatment, and outcome and pointing to possible causes for these disparities.”
Algorithmic Recourse: from Counterfactual Explanations to Interventions (2020, background): “assumptions when the true SCM is not available [2, 5, 19, 23, 44].”
The false promise of risk assessments: epistemic reform and the limits of fairness (2020, background): “Criminal justice institutions can change what interventions are made based on risk assessments, responding to risk with support rather than punishment [8, 62, 108].”
Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI (2020, background): “[6] Chelsea Barabas, Madars Virza, Karthik Dinakar, Joichi Ito, and Jonathan Zittrain.”
A Hierarchy of Limitations in Machine Learning (2020, methodology): “Barabas et al. (2018) discuss how risk assessments in the criminal justice system are also used for diagnostic purposes and in deciding how the criminal justice system intervenes in the lives of offenders; but this is unreliable.”
Algorithmic Fairness: Choices, Assumptions, and Definitions (2021, background): “In particular, causal graphs can be used to develop interventions at earlier points, prior to decision-making ( Jackson & VanderWeele 2018, Barabas et al. 2018).”
What About Applied Fairness? (2018, background): “The attention and critical thought is well deserved as we see applications emerge which can dramatically impact people’s lives and families, such as predictive policing (Ensign et al., 2018) and sentencing (Choulde-chova, 2017; Barabas et al., 2018).”
Fairness in Machine Learning: A Survey (2020, unknown): “[118, 269, 49, 72, 15, 33, 96, 217, 297, 41, 42, 203, 181]), governments (e.”
Fairness through Causal Awareness: Learning Causal Latent-Variable Models for Biased Data (2018, methodology): “t”) to be taken in the world, which in turn yields an outcome. In other words, the central question is how to properly intervene in an ongoing and evolving process, rather than predict outcomes alone [4]. Decision-making, i.e. learning to intervene, requires a fundamentally diﬀerent approach from le”
Counterfactual risk assessments, evaluation, and fairness (2019, background): “Prior literature has raised significant concerns regarding the fairness, transparency, and effectiveness of existing RAIs [3, 4, 9, 10, 13].”
This Thing Called Fairness (2019, background): “gaging cutting edge research in cryptography [84], formal languages [1], and systems [63, 118]. Research by social scientists and humanitiesscholarshassimilarlyadvancedthestate oftheoryandapplication [5, 11,80,136,149]. But like other eﬀorts to research, and address values in an integrative fashion,”
A Broader View on Bias in Automated Decision-Making: Reflecting on Epistemology and Dynamics (2018, background): “Barabas et al. argue that rather than prediction, “machine learning models should be used to surface covariates that are fed into a causal model for understanding the social, structural and psychological drivers of crime” (Barabas et al., 2018).”