Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness

By Michael Kearns, Seth Neel, Aaron Roth, Zhiwei Steven Wu. 2017

917 citations recorded by the source corpus.

The most prevalent notions of fairness in machine learning are statistical definitions: they fix a small collection of pre-defined groups, and then ask for parity of some statistic of the classifier across these groups. Constraints of this form are susceptible to intentional or inadvertent "fairness gerrymandering", in which a classifier appears to be fair on each individual group, but badly violates the fairness constraint on one or more structured subgroups defined over the protected attributes. We propose instead to demand statistical notions of fairness across exponentially (or infinitely) many subgroups, defined by a structured class of functions over the protected attributes. This interpolates between statistical definitions of fairness and recently proposed individual notions of fairness, but raises several computational challenges. It is no longer clear how to audit a fixed classifier to see if it satisfies such a strong definition of fairness. We prove that the computational problem of auditing subgroup fairness for both equality of false positive rates and statistical parity is equivalent to the problem of weak agnostic learning, which means it is computationally hard in the worst case, even for simple structured subclasses. We then derive two algorithms that provably converge to the best fair classifier, given access to oracles which can solve the agnostic learning problem. The algorithms are based on a formulation of subgroup fairness as a two-player zero-sum game between a Learner and an Auditor. Our first algorithm provably converges in a polynomial number of steps. Our second algorithm enjoys only provably asymptotic convergence, but has the merit of simplicity and faster per-step computation. We implement the simpler algorithm using linear regression as a heuristic oracle, and show that we can effectively both audit and learn fair classifiers on real datasets.

Subfields: Safety Research, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing (2020, background): “This is an observation of the fairness gerrymandering effect [28] - where optimizing for fairness on one axis can compromise fairness in another.”
Beyond Individual and Group Fairness (2020, methodology): “…of recent research work on the topic of fairness and algorithmic bias in machine learning [Dwork et al., 2012, Zemel et al., 2013, Hardt et al., 2016, Kleinberg et al., 2017, Pleiss et al., 2017, Agarwal et al., 2018, Kearns et al., 2018, Gillen et al., 2018, Sharifi-Malvajerdi et al., 2019].”
Differential Privacy Has Disparate Impact on Model Accuracy (2019, background): “Prior work on subgroup fairness aims to achieve good accuracy on all subgroups [21] using agnostic learning [22, 29].”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “Kearns et al. apply approximate statistical and false positive rate parity to groups of large enough size that are deﬁned by the sensitive variables [71].”
Fairness Without Demographics in Repeated Loss Minimization (2018, background): “Kearns et al. (2018); H ´ ebert-Johnson et al. (2017) consider subgroups of a set of protected features, but deﬁning non-trivial protected features which cover the latent demographics in our setting is dif-ﬁcult.”
Learning Adversarially Fair and Transferable Representations (2018, background): “An intriguing body of recent work unifies the individual-group dichotomy by exploring fairness at the intersection of multiple group identities, and among small subgroups of individuals (Kearns et al., 2018; Hébert-Johnson et al., 2018).”
A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual &Group Unfairness via Inequality Indices (2018, background): “When the number of groups is small and people within each group receive highly unequal benefits, the contribution of the between-group component to overall unfairness is small, and narrowing down attention to reducing Iβ alone may result in fairness gerrymandering [22]: while it is often easy to red”
Fairness Through Computationally-Bounded Awareness (2018, background): “Metric multifairness does not directly generalize either [14] or [16], but we argue that it provides a more flexible alternative to these approaches for subpopulation fairness.”
Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, background): “[23] similarly require repeated calls or knowledge of the model.”
Wasserstein Fair Classification (2019, methodology): “We pre-processed the data as in Wu et al. (2018).”
Flexibly Fair Representation Learning by Disentanglement (2019, neutral): “subgroup fair classification, Kearns et al. (2018); HebertJohnson et al.”
Compositional Fairness Constraints for Graph Embeddings (2019, background): “Note that this does not necessarily imply “subgroup fairness” on the joint distribution (Kearns et al., 2017).”
Proportionally Fair Clustering (2019, methodology): “gal system to the quality of recommender systems, researchers have devoted considerable eort to developing fair algorithms for the canonical supervised learning tasks of classication and regression [13, 28, 20, 27, 34, 11, 30, 35, 26, 18, 21]. We extend this work to a canonical problem in unsupervis”
50 Years of Test (Un)fairness: Lessons for Machine Learning (2018, background): “, skin tone), as recent work has begun to address [27, 39].”
The Frontiers of Fairness in Machine Learning (2018, background): “Two recent papers, [KNRW18b] and [HJKRR18] (see also [KNRW18a, KGZ18] for empirical evaluations of the algorithms proposed in these papers), attempt to do this by asking for statistical fairness deﬁnitions to hold not just on a small number of protected groups, but on an exponential or inﬁnite…”