On the Direction of Discrimination: An Information-Theoretic Analysis of Disparate Impact in Machine Learning
By Hao Wang, Berk Ustun, Flávio du Pin Calmon. 2018
12 citations recorded by the source corpus.
In the context of machine learning, disparate impact refers to a form of systematic discrimination whereby the output distribution of a model depends on the value of a sensitive attribute (e.g., race or gender). In this paper, we propose an information-theoretic framework to analyze the disparate impact of a binary classification model. We view the model as a fixed channel, and quantify disparate impact as the divergence in output distributions over two groups. Our aim is to find a correction function that can perturb the input distributions of each group to align their output distributions. We present an optimization problem that can be solved to obtain a correction function that will make the output distributions statistically indistinguishable. We derive closed-form expressions to efficiently compute the correction function, and demonstrate the benefits of our framework on a recidivism prediction problem based on the ProPublica COMPAS dataset.
Subfields: Artificial Intelligence, Safety Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Safety Research
Citations in the corpus
- Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, methodology): “[34] trained a model to predict outcomes and another to predict membership in a protected subgroup.”
- Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions (2019, background): “Distribution Alignment is a metric proposed in Wang et al. (2018) that measures the disparity of predicted outcomes via the KL-divergence, and is related to the statistical indistinguishablility of predictions between sensitive group. regulations on disparate treatment in applications such as…”
- Auditing Black-Box Models Using Transparent Model Distillation With Side Information (2017, methodology): “[43] train a model to predict outcomes and another to predict membership in a protected subgroup, then use the connection between the two models to identify features that proxy for protected features.”