Fairness without Demographics through Adversarially Reweighted Learning

By Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, Ed H. Chi. Neural Information Processing Systems, 2020

399 citations recorded by the source corpus.

Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns. However, in practice factors like privacy and regulation often preclude the collection of protected features, or their use for training or inference, severely limiting the applicability of traditional fairness research. Therefore we ask: How can we train a ML model to improve fairness when we do not even know the protected group memberships? In this work we address this problem by proposing Adversarially Reweighted Learning (ARL). In particular, we hypothesize that non-protected features and task labels are valuable for identifying fairness issues, and can be used to co-train an adversarial reweighting approach for improving fairness. Our results show that ARL improves Rawlsian Max-Min fairness, with significant AUC improvements for worst-case protected groups in multiple datasets,outperforming state-of-the-art alternatives.

Subfields: Artificial Intelligence, Safety Research

View Paper: https://www.semanticscholar.org/paper/08cb7d416bcd1b1e4d6492a0cd0b01424abd9515

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

Robust Optimization for Fairness with Noisy Protected Groups (2020, neutral): “[42] refer to as Rawlsian Max-Min fairness.”
Environment Inference for Invariant Learning (2020, methodology): “We ﬁnd that ARL contains noticeable discrepancies in the calibration curves across groups indicating that subgroup sufﬁciency has not been achieved.”
DORO: Distributional and Outlier Robust Optimization (2021, background): “The high-level idea of DORO can be extended to other algorithms that deal with subpopulation shift, such as static reweighting (Shimodaira, 2000), adversarial reweighting (Hu et al., 2018; Lahoti et al., 2020) and group DRO (Sagawa et al., 2020a).”
Bias in medical AI: Implications for clinical decision-making (2024, background): “During model training, the loss function can be modified to assign higher weights to samples from underrepresented groups, encouraging the model to prioritize correct predictions for these samples [104].”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “Ad-versarially Re-weighted Learning (Lahoti et al. 2020) aims to address this issue by relying on measurable covariates of protected characteristics (e.g., zip code as a proxy for race).”
A Survey on the Fairness of Recommender Systems (2022, background): “In practice, some work on classification tasks has also found that improving fairness may improve overall accuracy [50].”
Data collection and quality challenges in deep learning: a data-centric AI perspective (2021, background): “3 Tabular & Image Fairness-oriented [74,97,98,168]”
No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems (2020, background): “[29] Preeti Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi.”
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks (2021, neutral): “[43] consider an adversarially reweighted approach.”