Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment
By Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, Krishna P. Gummadi. 2016
1,325 citations recorded by the source corpus.
Automated data-driven decision making systems are increasingly being used to assist, or even replace humans in many settings. These systems function by learning from historical decisions, often taken by humans. In order to maximize the utility of these systems (or, classifiers), their training involves minimizing the errors (or, misclassifications) over the given historical data. However, it is quite possible that the optimally trained classifier makes decisions for people belonging to different social groups with different misclassification rates (e.g., misclassification rates for females are higher than for males), thereby placing these groups at an unfair disadvantage. To account for and avoid such unfairness, in this paper, we introduce a new notion of unfairness, disparate mistreatment, which is defined in terms of misclassification rates. We then propose intuitive measures of disparate mistreatment for decision boundary-based classifiers, which can be easily incorporated into their formulation as convex-concave constraints. Experiments on synthetic as well as real world datasets show that our methodology is effective at avoiding disparate mistreatment, often at a small cost in terms of accuracy.
Subfields: Safety Research, Sociology and Political Science, Statistics and Probability, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Safety Research
- Silvia Chiappa
- Luca Oneto
- Dana Pessach
- Erez Shmueli
- Adrian Weller
- Krishna P. Gummadi
- Aaron Roth
- Emilio Ferrara
In Sociology and Political Science
- Silvia Chiappa
- Krishna P. Gummadi
- Dana Pessach
- Erez Shmueli
- Aaron Roth
- Sharad Goel
- Sam Corbett-Davies
- Luca Oneto
In Statistics and Probability
- E.J. Candes
- Bernhard Schölkopf
- Adrian Weller
- Junzhe Zhang
- Stefan Bauer
- Rina Foygel Barber
- Emilio Ferrara
- Silvia Chiappa
In AI Ethics, Fairness & Society
- Silvia Chiappa
- Dana Pessach
- Erez Shmueli
- Luca Oneto
- Adrian Weller
- Berk Ustun
- Krishna P. Gummadi
- Lingxiao Huang
Citations in the corpus
- Does Object Recognition Work for Everyone? (2019, background): “The equalized odds [18] fairness principle (also referred to as disparate mistreatment [48]) requires classification algorithms to make predictions such that no group receives a disproportionately higher number of false-positive or false-negative errors.”
- Aligning AI With Shared Human Values (2020, background): “Research in algorithmic fairness initially began with simple statistical constraints (Lewis, 1978; Dwork et al., 2011; Hardt et al., 2016; Zafar et al., 2017), but these constraints were found to be mutually incompatible (Kleinberg et al., 2017) and inappropriate in many situations (CorbettDavies…”
- Counterfactual Fairness (2017, background): “These include fairness through unawareness [12], individual fairness [10, 16, 24, 38], demographic parity/disparate impact [36], and equality of opportunity [14, 37].”
- Fair Resource Allocation in Federated Learning (2019, background): “Another set of works optimize an objective subject to some fairness constraints during training time [3, 6, 18, 43, 46, 47, 9].”
- Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “call all four pairs of definitions (as well as equal accuracy) avoiding disparate mistreatment [130].”
- A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual &Group Unfairness via Inequality Indices (2018, background): “[40] to remove the between-group unfairness based on false negative rates between different races (Whites and non-Whites) in the COMPAS dataset.”
- Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction (2018, background): “Zafar et al. (2017) first tried to enforce equal False Positive and False Negative rates for groups by introducing penalties for misclassified data points during training.”
- Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations (2017, background): “, 2016] or by constraining models to avoid it [Dwork et al., 2012; Zafar et al., 2016]).”
- Proportionally Fair Clustering (2019, background): “The standard model in fair supervised learning [13, 28, 27, 35, 34] has a set of protected agents given as input to an algorithm which must classify agents into a positive and negative group.”
- Diversity in Faces (2019, methodology): “The gender annotations used two class labels (male and female) and the age group labeling used seven classes ([0-3],[4-12],[13-19],[20-30],[31-45],[46-60],[61-]), as well as a continuous age value to be consistent with the automatic prediction labels.”
- Aequitas: A Bias and Fairness Audit Toolkit (2018, methodology): “…methods to avoid bias and disparate impact or defining tradeoffs among different criteria (Calders and Verwer, 2010; Kamishima et al., 2011; Dwork et al., 2012; Zemel et al., 2013; Feldman et al., 2015; Hardt et al., 2016; Kleinberg et al., 2016; Corbett-Davies et al., 2017; Zafar et al., 2017b,a).”