Identifying Significant Predictive Bias in Classifiers

By Zhe Zhang, Daniel B. Neill. 2016

68 citations recorded by the source corpus.

We present a novel subset scan method to detect if a probabilistic binary classifier has statistically significant bias -- over or under predicting the risk -- for some subgroup, and identify the characteristics of this subgroup. This form of model checking and goodness-of-fit test provides a way to interpretably detect the presence of classifier bias or regions of poor classifier fit. This allows consideration of not just subgroups of a priori interest or small dimensions, but the space of all possible subgroups of features. To address the difficulty of considering these exponentially many possible subgroups, we use subset scan and parametric bootstrap-based methods. Extending this method, we can penalize the complexity of the detected subgroup and also identify subgroups with high classification errors. We demonstrate these methods and find interesting results on the COMPAS crime recidivism and credit delinquency data.

Subfields: Artificial Intelligence, Sociology and Political Science, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Statistics and Probability

Citations in the corpus

Fairness in Machine Learning: A Survey (2020, background): “Subgroup analysis can also be a useful exercise in model evaluation [2, 70, 315] often defining quantities to measure how models affect different subgroups; for example, to analyze if one model is more discriminatory than another to some observed subgroup, or to identify how variable omission (see…”
Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions (2019, background): “This may provide an interesting alternative to identify subgroups in which discrimination is most pronounced (see e.g, Chouldechova and G’Sell, 2017; Zhang and Neill, 2016).”
Optimized Pre-Processing for Discrimination Prevention (2017, background): “, 2016], as is calibration or lack of predictive bias in the predictions [Zhang and Neill, 2016].”