Fair decision making using privacy-protected data

By Satya Kuppam, Ryan McKenna, David Pujol, Michael Hay, Ashwin Machanavajjhala, Gerome Miklau. 2019

120 citations recorded by the source corpus.

Data collected about individuals is regularly used to make decisions that impact those same individuals. We consider settings where sensitive personal data is used to decide who will receive resources or benefits. While it is well known that there is a trade-off between protecting privacy and the accuracy of decisions, we initiate a first-of-its-kind study into the impact of formally private mechanisms (based on differential privacy) on fair and equitable decision-making. We empirically investigate novel tradeoffs on two real-world decisions made using U.S. Census data (allocation of federal funds and assignment of voting rights benefits) as well as a classic apportionment problem. Our results show that if decisions are made using an ∈-differentially private version of the data, under strict privacy constraints (smaller ∈), the noise added to achieve privacy may disproportionately impact some groups over others. We propose novel measures of fairness in the context of randomized differentially private algorithms and identify a range of causes of outcome disparities. We also explore improved algorithms to remedy the unfairness observed.

Subfields: Artificial Intelligence, Economics and Econometrics, Sociology and Political Science, Privacy & Security, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Eugene Bagdasarian
2. Vitaly Shmatikov
3. Vitaly Feldman
4. Reza Shokri
5. Congzheng Song
6. Prateek Mittal
7. Florian Tramèr
8. Lingjuan Lyu

In Sociology and Political Science

1. Vitaly Shmatikov
2. Eugene Bagdasarian
3. Arjun Narayan
4. Jihun Hamm
5. Ninghui Li
6. Cynthia Dwork
7. Manas A. Pathak

In Privacy & Security

1. Vitaly Shmatikov
2. Eugene Bagdasarian
3. Arjun Narayan
4. Aleksandra Korolova
5. Manas A. Pathak

In AI Ethics, Fairness & Society

1. Vitaly Shmatikov

Citations in the corpus

• Differential Privacy Has Disparate Impact on Model Accuracy (2019, background): “In concurrent work, Kuppam et al. [23] show that resource allocation based on DP statistics can disproportionately affect some subgroups.”
• Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels (2021, unknown): “, 2020), which can have financial and societal ramifications (Pujol et al., 2020).”