Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment

By Laurel Eckhouse, Kristian Lum, Cynthia Conti-Cook, J. Ciccolini. 2018

144 citations recorded by the source corpus.

Scholars in several fields, including quantitative methodologists, legal scholars, and theoretically oriented criminologists, have launched robust debates about the fairness of quantitative risk assessment. As the Supreme Court considers addressing constitutional questions on the issue, we propose a framework for understanding the relationships among these debates: layers of bias. In the top layer, we identify challenges to fairness within the risk-assessment models themselves. We explain types of statistical fairness and the tradeoffs between them. The second layer covers biases embedded in data. Using data from a racially biased criminal justice system can lead to unmeasurable biases in both risk scores and outcome measures. The final layer engages conceptual problems with risk models: Is it fair to make criminal justice decisions about individuals based on groups? We show that each layer depends on the layers below it: Without assurances about the foundational layers, the fairness of the top layers is irrelevant.

Subfields: Economics and Econometrics, Sociology and Political Science, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Sociology and Political Science

In Safety, Robustness & Interpretability

In AI Ethics, Fairness & Society

Citations in the corpus

Measurement and Fairness (2019, background): “[19] speak to “layers of bias” in risk assessment models: “we show that each layer depends on the layers below it: Without assurances about the foundational layers, the fairness of the top layers is irrelevant.”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “Several recent papers have demonstrated how social goals are, sometimes clumsily, abstracted and formulated to fit into a prediction task [23, 114, 48, 49, 94, 29, 113, 32, 100].”
A Causal Bayesian Networks Viewpoint on Fairness (2018, neutral): “In the reminder of the manuscript, we use this framework more extensively to further advance our analysis on fairness.”
Evaluation Gaps in Machine Learning Practice (2022, background): “Changes to the ecosystem itself—e.g., addressing what social change is perceived as possible and desirable [49, 68, 79]—are assumed to be out of scope, as are concerns for setting of precedents for other ML developers.”
Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance (2022, background): “Similar dynamics have been found in credit rating agencies [20, 40], supply chain audits [69], and accounting audits [58].”
The false promise of risk assessments: epistemic reform and the limits of fairness (2020, background): “1In practice, risk assessments are based in data about arrests, which typically represents a racially biased measure of crime [45, 108].”