Multiaccuracy : Black-Box Post-Processing for Fairness in Classi cation

By Michael P. Kim. 2018

Subfields: AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In AI Ethics, Fairness & Society

Citations in the corpus

Fairness Through Computationally-Bounded Awareness (2018, background): “Post-processing prediction models for fairness has been studied in a few contexts [25, 14, 18].”
The Frontiers of Fairness in Machine Learning (2018, background): “Two recent papers, [KNRW18b] and [HJKRR18] (see also [KNRW18a, KGZ18] for empirical evaluations of the algorithms proposed in these papers), attempt to do this by asking for statistical fairness deﬁnitions to hold not just on a small number of protected groups, but on an exponential or inﬁnite…”
An Empirical Study of Rich Subgroup Fairness for Machine Learning (2018, background): “Kim et al. [2018a] extend this style of analysis to accuracy constraints (asking that a classi-ﬁer be equally accurate on a combinatorially large collection of subgroups).”
Characterizing Fairness Over the Set of Good Models Under Selective Labels (2021, neutral): “Post-processing techniques typically modify the predictions from an existing model to achieve a target notion of fairness [26, 30, 31].”
Distilling Model Failures as Directions in Latent Space (2022, unknown): “Moreover, identifying meaningful subpopulations within data allows for dataset reﬁnement (such as ﬁltering or relabeling) (Yang et al., 2019; Stock & Cisse, 2018), and training more fair (Kim et al., 2019; Du et al., 2021) or accurate (Jabbour et al., 2020; Srivastava et al., 2020) models.”
Average Individual Fairness: Algorithms, Generalization and Experiments (2019, background): “This line includes Hébert-Johnson et al. (2018) (focusing on calibration), Kearns et al. (2018) (focusing on false positive and negative rates), and Kim et al. (2018b) (focusing on error rates).”
Fairness in Machine Learning: A Survey (2020, unknown): “…approaches such as tensor-based recommendations have been proposed that take statistical parity into account [317] and a mechanism design approach for fairly dividing a set of goods between groups using disparate impact as fairness measure and a recommender system as evaluation use case [235].”
Just Train Twice: Improving Group Robustness without Training Group Information (2021, background): “Kim et al. (2019) propose an auditing scheme that searches for high-loss groups deﬁned by a function within a pre-speciﬁed complexity class and postprocess the model to minimize discrepancies identiﬁed by the auditor.”
Conformal prediction with conditional guarantees (2023, unknown): “In particular, we show how our method can be used to satisfy two popular coverage desiderata: group-conditional coverage and coverage under covariate shift.”
With Malice Towards None: Assessing Uncertainty via Equalized Coverage (2019, background): “A major area of study has been to propose mathematical deﬁnitions of appropriate notions of fairness [1, 7–11] or algorithmic models of fairness [12].”
FairFed: Enabling Group Fairness in Federated Learning (2021, unknown): “…group fairness can be classiﬁed into three categories: pre-processing (Grgi ´ c-Hla ˇ ca et al. 2018; Feldman et al. 2015), in-processing (Kamishima et al. 2012; Zhang, Lemoine, and Mitchell 2018; Roh et al. 2021) and post-processing (Lohia et al. 2019; Kim, Ghorbani, and Zou 2019) techniques.”
Multicalibration: Calibration for the (Computationally-Identifiable) Masses (2018, background): “…how to interpolate between statistical and individual notions of “metric fairness” for general similarity metrics (Kim et al., 2018b), as well as further theoretical and empirical investigations of multi-accuracyin-expectation in the context of binary classification (Kim et al., 2018a).”
Towards Out-Of-Distribution Generalization: A Survey (2021, background): “[192], [213], [214] also adopt adversarially learning methods to ensure all computationally identiﬁable subgroups are treated equally.”
Fairness in Machine Learning (2020, methodology): “Post-Processing methods consist in transforming the model outputs in order to make them fair [2, 7, 31, 42, 51, 54, 68, 69, 99, 100, 106, 137, 147, 153, 156, 178].”
Your fairness may vary: Pretrained language model fairness in toxic text classification (2021, background): “…boundary for the protected group (Fish et al., 2016), iterative post-processing to achieve unbiased predictions on every identiﬁable subpopulation (Kim et al., 2019), recalibrating a classiﬁer us-ing a group-dependent threshold to optimize equality of opportunity (deﬁned as the difference…”
FrugalML: How to Use ML Prediction APIs More Accurately and Cheaply (2020, background): “For example, accuracy disparities in gender classiﬁcation were observed for different skin colors [22, 33].”
Environment Inference for Invariant Learning (2020, methodology): “We aim to overcome the difﬁculty of manual environment speciﬁcation by developing a new method inspired by fairness approaches for unknown group memberships (Kim et al., 2019; Lahoti et al., 2020).”