Eliciting and Enforcing Subjective Individual Fairness

By Christopher Jung, Michael Kearns, Seth Neel, Aaron Roth, Logan Stapleton, Zhiwei Steven Wu. 2019

56 citations recorded by the source corpus.

We revisit the notion of individual fairness first proposed by Dwork et al. [2012], which asks that "similar individuals should be treated similarly". A primary difficulty with this definition is that it assumes a completely specified fairness metric for the task at hand. In contrast, we consider a framework for fairness elicitation, in which fairness is indirectly specified only via a sample of pairs of individuals who should be treated (approximately) equally on the task. We make no assumption that these pairs are consistent with any metric. We provide a provably convergent oracle-efficient algorithm for minimizing error subject to the fairness constraints, and prove generalization theorems for both accuracy and fairness. Since the constrained pairs could be elicited either from a panel of judges, or from particular individuals, our framework provides a means for algorithmically enforcing subjective notions of fairness. We report on preliminary findings of a behavioral study of subjective fairness using human-subject fairness constraints elicited on the COMPAS criminal recidivism dataset.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “Recent work bypasses explicit elicitation of m and instead queries individuals only on whether |δ(v)− δ(v′)| is small for many pairs of individuals i and i′ [62].”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, background): “…in response, have proposed fairness metrics that attempt to use causality (Chiappa and Isaac 2019; Mitchell et al. 2018; Nabi and Shpitser 2018; Madras et al. 2019) or interactivity (Canetti et al. 2019; Jung et al. 2019) to integrate more contextual awareness of human conceptions of fairness.”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “Religion, disability status, and class are additional examples where fairness is often challenged by observability (Kattari, Olzman, and Hanna 2018; Sanchez and Schlossberg 2001).”
Decolonizing Content Moderation: Does Uniform Global Community Standard Resemble Utopian Equality or Western Power Hegemony? (2023, background): “Second, the platforms need to adopt fairness-aware computing approaches to learn how historical biases shape moderation outcomes [89] and enact participatory design practices by allowing stakeholders, who are directly impacted by moderation outcomes, to defne fairness in their own terms [73].”
Avoiding Resentment Via Monotonic Fairness (2019, background): “Recent concurrent works [25, 23] have explored the concept of estimating a distance metric by polling fair experts on what constitutes similarity.”
Individual Fairness in Pipelines (2020, background): “There is a wide variety of work concerning fairness in machine learning [9, 14, 25, 17, 3, 18, 26, 22, 10, 16, 15, 11, 12, 20, 19, 4, 24, 5].”
On the Legal Compatibility of Fairness Definitions (2019, background): “ML fairness literature is often presented as a step toward reducing algorithmic bias in high-stakes human systems—from the COMPAS risk assessment tool as part of the broader judicial system [27] to hypothetical loan mis-allocation scenarios affecting human livelihoods [22].”
Algorithmic Fairness: Choices, Assumptions, and Definitions (2021, background): “Incorrectly assuming that a sample is representative can lead to biased estimation of conditional probabilities (e.g., probability of crime given covariates), biased estimation of utility, and inadequate fairness adjustments (Kallus & Zhou 2018).”
On the apparent conflict between individual and group fairness (2019, background): “[18, 33, 36]).”