Multiaccuracy: Black-Box Post-Processing for Fairness in Classification

By Michael P. Kim, Amirata Ghorbani, James Y. Zou. 2018

404 citations recorded by the source corpus.

Prediction systems are successfully deployed in applications ranging from disease diagnosis, to predicting credit worthiness, to image recognition. Even when the overall accuracy is high, these systems may exhibit systematic biases that harm specific subpopulations; such biases may arise inadvertently due to underrepresentation in the data used to train a machine-learning model, or as the result of intentional malicious discrimination. We develop a rigorous framework of *multiaccuracy* auditing and post-processing to ensure accurate predictions across *identifiable subgroups*. Our algorithm, MULTIACCURACY-BOOST, works in any setting where we have black-box access to a predictor and a relatively small set of labeled data for auditing; importantly, this black-box framework allows for improved fairness and accountability of predictions, even when the predictor is minimally transparent. We prove that MULTIACCURACY-BOOST converges efficiently and show that if the initial model is accurate on an identifiable subgroup, then the post-processed model will be also. We experimentally demonstrate the effectiveness of the approach to improve the accuracy among minority subgroups in diverse applications (image classification, finance, population health). Interestingly, MULTIACCURACY-BOOST can improve subpopulation accuracy (e.g. for "black women") even when the sensitive features (e.g. "race", "gender") are not given to the algorithm explicitly.

Subfields: Artificial Intelligence, Health Informatics, Evaluation & Benchmarks, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Evaluation & Benchmarks

In AI Ethics, Fairness & Society

Citations in the corpus

Fairness Through Computationally-Bounded Awareness (2018, background): “Post-processing prediction models for fairness has been studied in a few contexts [25, 14, 18].”
Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, background): “’s iterative procedure [24] not only uncovers such regions but also modifies the model to improve accuracy in these regions.”
Flexibly Fair Representation Learning by Disentanglement (2019, methodology): “Building on the multi-accuracy framework of Hebert-Johnson et al. (2018), Kim et al. (2019) develop a new algorithm to achieve multi-group accuracy via a post-processing boosting procedure.”
An Empirical Study of Rich Subgroup Fairness for Machine Learning (2018, background): “extend this style of analysis to accuracy constraints (asking that a classifier be equally accurate on a combinatorially large collection of subgroups) [11].”
Characterizing Fairness Over the Set of Good Models Under Selective Labels (2021, methodology): “Post-processing techniques typically modify an existing model to achieve a target notion of fairness [26, 30, 31].”
Environment Inference for Invariant Learning (2020, methodology): “Algorithmic fairness Our work draws inspiration from a rich body of recent work on learning fair classifiers in the absence of demographic labels (Hébert-Johnson et al., 2018; Kearns et al., 2018; Hashimoto et al., 2018; Kim et al., 2019; Lahoti et al., 2020).”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, result): “Morina et al. [96] propose a post-processing approach that generalizes that of Hardt et al. [58], and Kim et al. [79] similarly propose a post-processing approach as well as auditing procedure.”
Just Train Twice: Improving Group Robustness without Training Group Information (2021, background): “Kim et al. (2019) propose an auditing scheme that searches for high-loss groups deﬁned by a function within a pre-speciﬁed complexity class and postprocess the model to minimize discrepancies identiﬁed by the auditor.”
Multicalibration: Calibration for the (Computationally-Identifiable) Masses (2018, background): “…how to interpolate between statistical and individual notions of “metric fairness” for general similarity metrics (Kim et al., 2018b), as well as further theoretical and empirical investigations of multi-accuracyin-expectation in the context of binary classification (Kim et al., 2018a).”
Advances, challenges and opportunities in creating data for trustworthy AI (2022, methodology): “Multi-accuracy auditing is an algorithmic approach that automates the search for data subgroups where the AI model performs poorly 83 .”
FrugalML: How to Use ML Prediction APIs More Accurately and Cheaply (2020, background): “For example, accuracy disparities in gender classification were observed for different skin colors [22, 33].”
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (2023, methodology): “We use a proprietary dataset used to train ChatGPT RMs.”
Average Individual Fairness: Algorithms, Generalization and Experiments (2019, background): “This line includes Hébert-Johnson et al. (2018) (focusing on calibration), Kearns et al. (2018) (focusing on false positive and negative rates), and Kim et al. (2018b) (focusing on error rates).”
Fairness without Demographics through Adversarially Reweighted Learning (2020, background): “Computational-Identifiability: Related to our algorithm, a number of works [33, 35, 23, 36] address intersectional fairness by optimizing for group fairness between all computationally identifiable groups in the input space.”
Your fairness may vary: Pretrained language model fairness in toxic text classification (2021, background): “…boundary for the protected group (Fish et al., 2016), iterative post-processing to achieve unbiased predictions on every identiﬁable subpopulation (Kim et al., 2019), recalibrating a classiﬁer us-ing a group-dependent threshold to optimize equality of opportunity (deﬁned as the difference…”