Why Is My Classifier Discriminatory?

By Irene Y. Chen, Fredrik D. Johansson, David A. Sontag. 2018

448 citations recorded by the source corpus.

Recent attempts to achieve fairness in predictive models focus on the balance between fairness and accuracy. In sensitive applications such as healthcare or criminal justice, this trade-off is often undesirable as any increase in prediction error could have devastating consequences. In this work, we argue that the fairness of predictions should be evaluated in context of the data, and that unfairness induced by inadequate samples sizes or unmeasured predictive variables should be addressed through data collection, rather than by constraining the model. We decompose cost-based metrics of discrimination into bias, variance, and noise, and propose actions aimed at estimating and reducing each term. Finally, we perform case-studies on prediction of income, mortality, and review ratings, confirming the value of this analysis. We find that data collection is often a means to reduce discrimination without sacrificing accuracy.

Subfields: Health Information Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health Information Management

Citations in the corpus

Fair Regression: Quantitative Definitions and Reduction-based Algorithms (2019, background): “They might point to data issues requiring non-algorithmic interventions, such as gathering of additional (less biased) data or introduction of new features (Chen et al., 2018).”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (2018, methodology): “For example, while the fair ML literature has largely focused on “de-biasing” algorithms, which view the training data as fixed [21, 55], most of our interviewees”
Incorporating Priors with Feature Attribution on Text Classification (2019, background): “Instead, researchers try to reduce biases indirectly by mostly adding more data (Dixon et al., 2018; Chen et al., 2018), using unbiased word vectors (Park et al., 2018), or directly optimizing for a fairness proxy with adversarial training (Madras et al., 2018; Zhang et al., 2018a).”
On the Accuracy of Influence Functions for Measuring Group Effects (2019, methodology): “, 2010) or demographic groups (Chen et al., 2018); apportion credit between crowdworkers, each of whom generated part of the data (Arrieta-Ibarra et al.”
Lessons for artificial intelligence from the study of natural stupidity (2019, background): “Recent work shows that collecting more data about rare groups can reduce algorithmic discrimination 40 , but data collection can be expensive or infeasible.”
Ethical Machine Learning in Healthcare (2020, methodology): “While there is scholarship addressing social implications and algorithmic fairness in general, there has been less work at the intersection of health, ML, and fairness (14, 15, 16), despite the potential life-or-death impacts of those models (8, 17).”
Fairness without Harm: Decoupled Classifiers with Preference Guarantees (2019, background): “In comparison, less work has sought to articulate suitable notions of fairness for domains with different ethical principles (with some exceptions, see e.g., Chen et al., 2018).”
Model Multiplicity: Opportunities, Concerns, and Solutions (2022, background): “The existence of model multiplicity presents exciting opportunities because it offers model makers the flexibility to prioritize, and optimize for, desirable properties at no cost to accuracy, contrary to some conventionalwisdom [7, 16, 28, 42, 57, 60, 62].”
Fairness through Causal Awareness: Learning Causal Latent-Variable Models for Biased Data (2018, background): “[6] explore an idea which is easily adjusted to the interventional setting: that an algorithm is unfair if it is much less accurate on one subgroup.”
Fairness in Machine Learning: A Survey (2020, background): “Perhaps, as [60, 187, 94, 276] note, it may often be better to fix the underlying data sample (e.”
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2020, unknown): “richer datasets [30].”
Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions (2019, background): “Such disparities have led to a plethora of research on fair machine learning, focusing on how disparate impact arises (Chen et al., 2018; Datta et al., 2016), how it can be measured (Žliobaitė, 2017; Pierson et al.”
A snapshot of the frontiers of fairness in machine learning (2020, background): “This effect can be quantified and can be partially alleviated via concerted data gathering effort.(14) 3.”