Differentially Private Fair Learning

By Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi-Malvajerdi, Jonathan Ullman. 2018

174 citations recorded by the source corpus.

Motivated by settings in which predictive models may be required to be non-discriminatory with respect to certain attributes (such as race), but even collecting the sensitive attribute may be forbidden or restricted, we initiate the study of fair learning under the constraint of differential privacy. We design two learning algorithms that simultaneously promise differential privacy and equalized odds, a 'fairness' condition that corresponds to equalizing false positive and negative rates across protected groups. Our first algorithm is a private implementation of the equalized odds post-processing approach of [Hardt et al., 2016]. This algorithm is appealingly simple, but must be able to use protected group membership explicitly at test time, which can be viewed as a form of 'disparate treatment'. Our second algorithm is a differentially private version of the oracle-efficient in-processing approach of [Agarwal et al., 2018] that can be used to find the optimal fair classifier, given access to a subroutine that can solve the original (not necessarily fair) learning problem. This algorithm is more complex but need not have access to protected group membership at test time. We identify new tradeoffs between fairness, accuracy, and privacy that emerge only when requiring all three properties, and show that these tradeoffs can be milder if group membership may be used at test time. We conclude with a brief experimental evaluation.

Subfields: Artificial Intelligence, Safety Research, Statistics and Probability, Privacy & Security, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Statistics and Probability

In Privacy & Security

In AI Ethics, Fairness & Society

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “hip in sensitive groups in order to measure or ensure fair predictions are being made. While the trade-off between differential privacy and fairness has been investigated in the non-federated setting [214, 127], there has been little work on how (or whether) FL may be able to uniquely address concer”
Differential Privacy Has Disparate Impact on Model Accuracy (2019, background): “Models that generate artiﬁcial data points [11] from the existing data are incompatible with DP. Recent research [10, 20] aims to add fairness and DP to post-processing [17] and in-processing [2] algorithms.”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “Differentially private fair machine learning (Jagielski et al. 2019) could potentially provide such guarantees, simultaneously meeting the requirements of fairness, privacy and accuracy.”
What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness (2020, neutral): “[46], Kilbertus et al.”
On the Compatibility of Privacy and Fairness (2019, background): “[12] shows two algorithms that satisfy both differential privacy and equalized odds.”
Fairness without Demographics through Adversarially Reweighted Learning (2020, background): “They generally assume that the ML model has access to the protected-features, albeit in encrypted form via secure multi-party computation [50, 34, 25], or in a privacy preserving form by employing differentially private learning [50, 27].”
Towards Fair and Privacy-Preserving Federated Deep Models (2019, background): “Existing research on fairness mostly focuses on protecting sensitive attributes or reducing the variance of the prediction distribution across participants [6], [7].”
Noise-tolerant fair classification (2019, background): “Recently, Jagielski et al. (2018) explored preserving differential privacy (Dwork, 2006) while at the same time of preserving fairness constraints and achieving good accuracy.”
On the Legal Compatibility of Fairness Definitions (2019, background): “In ML literature, disparate treatment is often explained as making use of the protected attribute in the decision-making process [26], while disparate impact is understood as when “outcomes differ across subgroups (even unintentionally)” [32].”
A Review on Fairness in Machine Learning (2022, background): “Most of these studies have investigated the case of a centralized setting, in which all the dataset is held by a single party [10, 52, 100, 106, 153, 210].”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “Uploading prediction vectors to the server may not satisfy DP guarantees.”