The Measure and Mismeasure of Fairness

By Sam Corbett-Davies, Johann D. Gaebler, H. Nilforoshan, Ravi Shroff, Sharad Goel. 2018

229 citations recorded by the source corpus.

The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals.

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Measurement and Fairness (2019, neutral): “Prompted by this, many academic papers began to discuss the properties and shortcomings of different definitions of group fairness, with some papers demonstrating that it is impossible for a system like COMPAS to possess both error-rate balance and predictive parity when the probability of recidivis”
Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability (2018, methodology): “Will an algorithmic approach, based on these measures, perpetuate (or even magnify) the disadvantage? Or could we use algorithms to increase equity between groups? [1, 3] • Interpretability.”
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks (2021, background): “The “impossibility of fairness” principle 237 [71, 72, 73, 74] gets at the fact that in an unequal society, equal treatment is guaranteed to result in 238 unequal outcomes [67], making the thoughtful selection of fairness definitions and their associated 239 values all the more critical.”