Fairness Under Composition

By Cynthia Dwork, Christina Ilvento. 2018

135 citations recorded by the source corpus.

Algorithmic fairness, and in particular the fairness of scoring and classification algorithms, has become a topic of increasing social concern and has recently witnessed an explosion of research in theoretical computer science, machine learning, statistics, the social sciences, and law. Much of the literature considers the case of a single classifier (or scoring function) used once, in isolation. In this work, we initiate the study of the fairness properties of systems composed of algorithms that are fair in isolation; that is, we study fairness under composition. We identify pitfalls of naive composition and give general constructions for fair composition, demonstrating both that classifiers that are fair in isolation do not necessarily compose into fair systems and also that seemingly unfair components may be carefully combined to construct fair systems. We focus primarily on the individual fairness setting proposed in [Dwork, Hardt, Pitassi, Reingold, Zemel, 2011], but also extend our results to a large class of group fairness definitions popular in the recent literature, exhibiting several cases in which group fairness definitions give misleading signals under composition.

Subfields: Safety Research, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Sociology and Political Science

Citations in the corpus

Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (2018, background): “lti-component ML system, there is not always a clean mapping between performance metrics for an individual ML model and the system’s utility for users. System components may interact with one another [33, 85] and with other aspects of a system’s design in ways that can be difficult to predict absent”
Algorithmic Fairness (2020, background): “This field is sometimes referred to as fair pipelines [21, 45, 52, 69, 97].”
Equity Beyond Bias in Language Technologies for Education (2019, background): “Classiﬁers may trivially fail to maintain fairness properties when the output from one classiﬁer is used as input for another, for instance (Dwork and Ilvento, 2018), or even worsen disparate outcomes after iterating on algorithmic predictions over time (Liu et al., 2018).”
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting (2019, background): “Recently, Dwork and Ilvento [19] showed that fairness does not hold under composition, meaning that if two classifiers are individually fair according to some fairness metric, then the sequential use of these classifiers will not necessarily be fair according the same metric.”
Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements (2019, methodology): “When the prediction is treated as a continuous score or used in conjunction with other signals, it becomes harder to evaluate the effect on the user experience [16] and further research is needed in this direction.”
Economics, Fairness and Algorithmic Bias (2020, unknown): “Computer scientists refer to this as “fairness under composition” (Dwork and Ilvento, 2018).”
Individual Fairness in Pipelines (2020, methodology): “First we introduce the weighted sampling mechanism, as described in [6].”
Making Decisions that Reduce Discriminatory Impacts (2019, unknown): “Other recent works (Komiyama & Shimao, 2018; Elzayn et al., 2018; Dwork & Ilvento, 2018) consider related problems about social outcomes, allocating resources, and the effects of multiple discrimination-free predictors.”
Pretending Fair Decisions via Stealthily Biased Sampling (2019, background): “Several efforts have devoted to establish mathematical formulation of fairness [2, 3, 4] and proposed algorithms that meet the fairness criteria [5, 6, 7].”
A Review on Fairness in Machine Learning (2022, background): “This field is sometimes referred to as fair pipelines [29, 61, 71, 99, 142].”