Multicalibration: Calibration for the (Computationally-Identifiable) Masses

By Úrsula Hébert-Johnson, Michael P. Kim, Omer Reingold, Guy N. Rothblum. 2018

504 citations recorded by the source corpus.

Subfields: AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In AI Ethics, Fairness & Society

Citations in the corpus

Flexibly Fair Representation Learning by Disentanglement (2019, methodology): “Building on the multi-accuracy framework of Hebert-Johnson et al. (2018), Kim et al. (2019) develop a new algorithm to achieve multi-group accuracy via a post-processing boosting procedure.”
Verified Uncertainty Calibration (2019, background): “Calibration, including the squared calibration error, has been studied in many fields besides machine learning including meteorology [2, 3, 4, 5, 6], fairness [29, 30], healthcare [1, 31, 32, 33], reinforcement learning [34], natural language processing [7, 8], speech recognition [35], econometrics ”
Individual Fairness in Pipelines (2020, background): “RELATED WORK There is a wide variety of work concerning fairness in machine learning [9, 14, 25, 17, 3, 18, 26, 22, 10, 16, 15, 11, 12, 20, 19, 4, 24, 5].”
Conformal prediction with conditional guarantees (2023, unknown): “Our relaxation is also referred to as a “multi-accuracy” objective in theoretical computer science (H´ebert-Johnson et al. (2018), Kim et al. (2019)).”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, background): “[59] learn a predictor for numerous overlapping demographic subgroups with a focus on calibration, and Foulds et al.”
Fairness without Harm: Decoupled Classifiers with Preference Guarantees (2019, methodology): “This reﬂects a notion that is robust against the possibility of “gerrymandering” along sensitive attributes (see e.g., Kearns et al., 2018; H´ebert-Johnson et al., 2018).”
The Implicit Fairness Criterion of Unconstrained Learning (2018, background): “Consequently, recent literature has proposed new learning algorithms which ensure approximate calibration in different settings (Hebert-Johnson et al., 2018; Kearns et al., 2017).”
Sex and gender analysis improves science and engineering (2019, methodology): “An alternative approach to quantify and reduce gender bias in algorithms is called multi-accuracy auditing 68,69 .”
On the Compatibility of Privacy and Fairness (2019, background): “More recently, differentially private tools contributed to the design of algorithms which are well-calibrated across many different group definitions simultaneously [11].”
With Malice Towards None: Assessing Uncertainty via Equalized Coverage (2019, background): “Here, forcing parity of true positive rates [8] would conﬂict with accurate predictions for each group [10].”
Average Individual Fairness: Algorithms, Generalization and Experiments (2019, background): “This line includes Hébert-Johnson et al. (2018) (focusing on calibration), Kearns et al. (2018) (focusing on false positive and negative rates), and Kim et al. (2018b) (focusing on error rates).”
Bayesian Modeling of Intersectional Fairness: The Variance of Bias (2018, background): “[17] and [15] proposed further related multi-attribute definitions regarding false positive rates and calibration, respectively.”
Directional Bias Amplification (2021, background): “Thus, matching the training correlations should not be the intended goal (Wick et al., 2019; Hebert-Johnson et al., 2018).”
An Intersectional Definition of Fairness (2018, background): “Other candidate multi-attribute fairness definitions include false positive subgroup fairness [27] and multicalibration [23].”