The (Im)possibility of fairness

By Sorelle A. Friedler, Carlos Eduardo Scheidegger, Suresh Venkatasubramanian. 2021

487 citations recorded by the source corpus.

What does it mean to be fair?

Subfields: Safety Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In AI Ethics, Fairness & Society

Citations in the corpus

Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs (2021, background): “However, in many scenarios, such labels may be inaccurate, either due to measurement issues [44] or due to discrimination [25].”
Measurement and Fairness (2019, background): “ations helps us understand the choices we are making and their consequences. Recalling our discussion of individual fairness, the construct of \equal opportunity" is nontrivial to operationalize [18, 22]. In fact, the complexity of the theoretical understandings of this construct is nontrivial:”
Predictability and Surprise in Large Generative Models (2022, background): “First, it is well known that there are many more fairness metrics than the two we consider here, and that it’s statistically impossible for a single algorithm to achieve parity on all these metrics e.g., [26].”
FairSight: Visual Analytics for Fairness in Decision Making (2019, methodology): “We adopt the Group Skew concept [12] to measure such bias as: where Distortion BTN = i ∈ S + ,j ∈ S − ,i (cid:54) = j distortion ( i, j ) and Distortion WTN = i,j ∈ S ( · ) ,i (cid:54) = j distortion ( i, j ) .”
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “Practitioners should be careful when adopting fairness metrics as each technical deﬁnition carry strong values assumptions which can often be in tension with each other (Friedler et al., 2016; Kleinberg et al., 2017).”
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification (2019, background): “rning models are being used for an ever-expanding set of use cases, many have raised concerns about the potential negative impact of unintended identity-based bias that could be present in the models [3, 8]. In recent years, significant research has been published presenting new metrics intended to ”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, neutral): “Friedler et al. (2016) deﬁned bias as unfairness in algorithms.”
Algorithmic Fairness (2020, background): “Interestingly, several recent studies have shown that it is not possible to satisfy multiple notions of fairness simultaneously [15, 36, 38, 39, 56, 85, 112].”
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks (2021, background): “The “impossibility of fairness” principle [73, 74, 75, 76] gets at the fact that in an unequal society, equal treatment is guaranteed to result in unequal outcomes [69], making the thoughtful selection of fairness definitions and their associated values all the more critical.”
Algorithmic Fairness: Choices, Assumptions, and Definitions (2021, background): “Friedler et al. (2016) introduce an assumption they call what you see is what you get (WYSIWYG), i.e., that we can define a metric in the observed space that approximates a metric in the construct space: m ( v i , v i (cid:2) ) ≈ m CS ( t i , t i (cid:2) ).”
On the Legal Compatibility of Fairness Definitions (2019, background): “Others have identified a gap between ML measurements and the complex reality of the socio-technical human environments in which they are deployed [35; 19].”
Fairness in representation: quantifying stereotyping as a representational harm (2019, background): “Friedler et al. [18] provide a framework that explicitly calls out the distinction be-tween the construct space , the desired representation of individuals, and the observed space , the measured attributes.”
Economics, Fairness and Algorithmic Bias (2020, unknown): “Formal proofs in Cowgill (2018c) and Friedler et al. (2016) suggest that an absolute removal in bias is exceedingly unlikely using real-world datasets.”
Auditing large language models: a three-layered approach (2023, background): “Moreover, different operationalisations of the same normative concept (like ‘fairness’) cannot be satisfied simultaneously [261].”
The Role of Protected Class Word Lists in Bias Identification of Contextualized Word Representations (2019, methodology): “NLP has begun tackling the problems that are limiting the achievement of fair and ethical AI (Hovy and Spruit, 2016; Friedler et al., 2016), including techniques for mitigating demographic biases in models.”
The Forgotten Margins of AI Ethics (2022, background): “It is also important to note that algorithmic fairness is a slippery notion that is understood in many different ways, typically with mutually incompatible conceptions [28].”