On the (im)possibility of fairness

By Sorelle A. Friedler, Carlos Eduardo Scheidegger, Suresh Venkatasubramanian. 2016

94 citations recorded by the source corpus.

What does it mean for an algorithm to be fair? Different papers use different notions of algorithmic fairness, and although these appear internally consistent, they also seem mutually incompatible. We present a mathematical setting in which the distinctions in previous papers can be made formal. In addition to characterizing the spaces of inputs (the "observed" space) and outputs (the "decision" space), we introduce the notion of a construct space: a space that captures unobservable, but meaningful variables for the prediction. We show that in order to prove desirable properties of the entire decision-making process, different mechanisms for fairness require different assumptions about the nature of the mapping from construct space to decision space. The results in this paper imply that future treatments of algorithmic fairness should more explicitly state assumptions about the relationship between constructs and observations.

Subfields: Artificial Intelligence, Management Science and Operations Research, Safety Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Safety Research

In AI Ethics, Fairness & Society

Citations in the corpus

Inherent Trade-Offs in the Fair Determination of Risk Scores (2016, background): “[15] defines two axiomatic pro pe ties of feature generation and shows that no mechanism can be fair under these two properties.”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “to define the construct space, the variables on which we want to base decisions [51].”
Fairness and Abstraction in Sociotechnical Systems (2019, background): “[26] discuss two: that the data observed is close enough to the facts that matter (WYSIWYG) or that groups are, on the whole, similar to each other with respect to the task, and thus "we’re all”
50 Years of Test (Un)fairness: Lessons for Machine Learning (2018, background): “in 2016 [28], Thorndike also pointed out the tension between individual notions of fairness and group notions of fairness: “the two definitions of fairness—one based on predicted criterion score for individuals and the other on the distribution of criterion scores in the two groups— will always be i”
Fairness in Criminal Justice Risk Assessments: The State of the Art (2017, background): “Instructive definitions can be found in computer science (Pedreschi, 2008; Kamishima et al., 2011; Dwork et al., 2012; Kamiran et al., 2012; Chouldechova, 2016; Friedler et al., 2016; Hardt et al., 2016; Joseph et al., 2016; Kleinberg et al., 2016; Calmon et al., 2017; Corbitt-Davies et al., 2017), ”
Fairness in Reinforcement Learning (2016, background): “More recently, Hardt et al. [9] proposed new notions of fairness and showed how to achieve these notions via post-processing of a black-box classiﬁer.”
From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices (2019, background): “…actually mean when there are currently a myriad definitions (Ananny and Crawford 2018; Bibal and Frénay 2016; Doshi-Velez and Kim 2017; Friedler et al. 2016; Guidotti et al. 2018; Kleinberg et al. 2016; Overdorf et al. 2018; Turilli and Floridi 2009) is standard fare for individuals…”
Fairness through Causal Awareness: Learning Causal Latent-Variable Models for Biased Data (2018, background): “Some of the canonical problems for which these algorithms have been proposed are loan assignment [15], criminal risk assessment [7], and school admissions [13].”
On the apparent conflict between individual and group fairness (2019, background): “The point above - that it is necessary to make contestable empirical and normative assumptions in order to pursue the aims of Fair-ML in a meaningful way - is closely related to arguments put forward in previous work by Friedler et al. [22].”
Measuring Fairness in Ranked Outputs (2016, background): “A useful dichotomy is between individual fairness — a requirement that similar individuals are treated similarly, and group fairness, also known as statistical parity — a requirement that demographics of those receiving a particular outcome are identical to the demographics of the population as a wh”
Designing Fair Ranking Schemes (2017, background): “come are identical or similar to the demographics of the population as a whole [6]. These two requirements represent intrinsically different world views, and accommodating both may require trade-offs [10]. Our focus is on group fairness, which is based on the relationship between (1) membership of i”
Human Decision Making with Machine Assistance (2019, background): “…delegated to machines [1, 4, 6, 31] and has extensively studied the distributive [26, 29, 39, 43, 49, 50, 56, 65, 73–75] and procedural [36, 38] fairness of machine decision-making [9, 14, 18, 19, 32, 46], as well as the feasibility of making fair decisions [14, 19, 32, 46], as reviewed in [9, 18].”
Gender shades : intersectional phenotypic and demographic evaluation of face datasets and gender classifiers (2017, methodology): “Conversely, techniques used to optimize for a WAE worldview that takes structural bias into account, do not function well when working on a construct space that can be viewed as representative of an inherent individual trait or objective performance (Friedler et al., 2016).”
Measuring and Mitigating Unintended Bias in Text Classification (2018, background): “[11] and [8] both compare several different fairness metrics.”