Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved

By Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, Madeleine Udell. 2018

270 citations recorded by the source corpus.

Assessing the fairness of a decision making system with respect to a protected class, such as gender or race, is challenging when class membership labels are unavailable. Probabilistic models for predicting the protected class based on observable proxies, such as surname and geolocation for race, are sometimes used to impute these missing labels for compliance assessments. Empirically, these methods are observed to exaggerate disparities, but the reason why is unknown. In this paper, we decompose the biases in estimating outcome disparity via threshold-based imputation into multiple interpretable bias sources, allowing us to explain when over- or underestimation occurs. We also propose an alternative weighted estimator that uses soft classification, and show that its bias arises simply from the conditional covariance of the outcome with the true class membership. Finally, we illustrate our results with numerical simulations and a public dataset of mortgage applications, using geolocation as a proxy for race. We confirm that the bias of threshold-based imputation is generally upward, but its magnitude varies strongly with the threshold chosen. Our new weighted estimator tends to have a negative bias that is much simpler to analyze and reason about.

Subfields: Artificial Intelligence, Safety Research, Strategy and Management, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Strategy and Management

In AI Ethics, Fairness & Society

Citations in the corpus

A Survey on Bias and Fairness in Machine Learning (2019, background): “e [79]. A list of protected or sensitive attributes is provided in Table 4. They point out what attributes should not affect the outcome of the decision in housing loan or credit card decision-making [29] according to the law. Some of the existing work tries to treat sensitive attributes as noise to”
Fair Transfer Learning with Missing Protected Attributes (2019, background): “Financial institutionscould also usethis to evaluatewhether their credit approval processes comply with regulations prohibiting discrimination in cases where they are legally or practically unable to collect the protected attribute [5].”
Awareness in practice: tensions in access to sensitive attribute data for antidiscrimination (2019, background): “sing techniques or directly implement fairness interventions. A significant body of research presumes the modeler has ready access to data on these characteristics as they build and test their models [13, 19, 21]. The need for this data is plain to see. As a 2003 analysis of racial disparities in he”
You Shouldn't Trust Me: Learning Models Which Conceal Unfairness From Multiple Explanation Methods (2020, background): “This is sometimes considered a form of process fairness via unawareness [8, 15].”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, background): “s are biased in their decision-making. Examples of cases in which XAI is used for bias mitigation and fairness assessment are criminal risk assessment [20, 115] and loan and insurance rate prediction [32]. It is worth mentioning that there is overlap between the biased decision-making mitigation goa”
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks (2021, background): “Unbiased recovery with covariate-based proxies is impossible [41].”
Fairness without Demographics through Adversarially Reweighted Learning (2020, background): “However, using proxies can in itself be prone to estimation bias [29, 10].”
Economics, Fairness and Algorithmic Bias (2020, unknown): “Some make precise adjustments to standard algorithms based on a model of the bias (Calders et al., 2009; Kamiran and Calders, 2009; Feldman et al., 2015; Agarwal et al., 2018; Chen et al., 2018b).”
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2020, background): “In contexts such as our running example, information on race and ethnicity of patients may not be collected [31].”