Fairness Violations and Mitigation under Covariate Shift

By Harvineet Singh, Rina Singh, Vishwali Mhasawade, Rumi Chunara. 2019

109 citations recorded by the source corpus.

We study the problem of learning fair prediction models for unseen test sets distributed differently from the train set. Stability against changes in data distribution is an important mandate for responsible deployment of models. The domain adaptation literature addresses this concern, albeit with the notion of stability limited to that of prediction accuracy. We identify sufficient conditions under which stable models, both in terms of prediction accuracy and fairness, can be learned. Using the causal graph describing the data and the anticipated shifts, we specify an approach based on feature selection that exploits conditional independencies in the data to estimate accuracy and fairness metrics for the test set. We show that for specific fairness definitions, the resulting model satisfies a form of worst-case optimality. In context of a healthcare task, we illustrate the advantages of the approach in making more equitable decisions.

Subfields: Artificial Intelligence, Health Informatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

Characterizing Fairness Over the Set of Good Models Under Selective Labels (2021, background): “Reweighing procedures have been proposed for jointly addressing covariate shift and fairness (Coston et al., 2019; Singh et al., 2021).”
Algorithm fairness in artificial intelligence for medicine and healthcare (2023, methodology): “…make strong assumptions about the learning scenario, such as the training and test data being independently and identically distributed, an assumption which is often violated when using data from hospitals in different geographies or when employing different data-curation protocols 86,104–107 .”
Coarse race data conceals disparities in clinical risk score performance (2023, background): “…example, the existence of covariate shift (differences in p(X)) between granular race groups suggests that models trained on certain granular groups may not generalize to others (Nestor et al., 2019), and that recent techniques to address covariate shift would be appropriate (Singh et al., 2021).”
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks (2021, background): “[58] study conditions 183 (assuming a given causal graph) under which a stable predictor, in the sense of accuracy and fairness, 184 could be learned.”