A survey on measuring indirect discrimination in machine learning

By Indrė Žliobaitė. 2015

156 citations recorded by the source corpus.

Nowadays, many decisions are made using predictive models built on historical data.Predictive models may systematically discriminate groups of people even if the computing process is fair and well-intentioned. Discrimination-aware data mining studies how to make predictive models free from discrimination, when historical data, on which they are built, may be biased, incomplete, or even contain past discriminatory decisions. Discrimination refers to disadvantageous treatment of a person based on belonging to a category rather than on individual merit. In this survey we review and organize various discrimination measures that have been used for measuring discrimination in data, as well as in evaluating performance of discrimination-aware predictive models. We also discuss related measures from other disciplines, which have not been used for measuring discrimination, but potentially could be suitable for this purpose. We computationally analyze properties of selected measures. We also review and discuss measuring procedures, and present recommendations for practitioners. The primary target audience is data mining, machine learning, pattern recognition, statistical modeling researchers developing new methods for non-discriminatory predictive modeling. In addition, practitioners and policy makers would use the survey for diagnosing potential discrimination by predictive models.

Subfields: Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (2016, background): “Feldman et al. [10] distinguish classification algorithms that achieve fairness by modifying the underlying data from those that achieve fairness by modifying the classification algorithm.”
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints (2017, background): “Various approaches have been proposed to make a “fair” binary classiﬁcation (Baro-cas and Selbst, 2014; Dwork et al., 2012; Feldman et al., 2015; Zliobaite, 2015).”
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems (2018, methodology): “on measuring discrimination in data as well as on evaluating performance of discrimination-aware predictive models (Zliobaite, 2015; Pedreshi et al., 2008; Hajian and Domingo-Ferrer, 2013; Goh et al., 2016).”
Inherent Trade-Offs in the Fair Determination of Risk Scores (2016, background): “Barocas and Selbst survey a range of ways in which data-analysis algorithms can lead to discriminatory outcomes [3], and review articles by Romei and Ruggieri [25] and Zliobaite [30] survey data-analytic and algorithmic methods for measuring discrimination.”
Counterfactual Fairness (2017, background): “There has been much recent interest in designing algorithms that make fair predictions [4, 6, 9, 10, 12, 14–17, 19, 21, 30–33].”
FairSight: Visual Analytics for Fairness in Decision Making (2019, background): “In fair ranking, early studies mainly focused on quantifying the discrimination with proposed ranking measures in the top-k list [32], or indirect discrimination [43].”
On formalizing fairness in prediction with machine learning (2017, background): “eople due to the membership to certain demographic groups that are distinguished by the a−ributes protected by law (henceforth, protected a‹ributes). A discriminatory action may be direct or indirect [81]. A direct discrimination is said to occur when a person is treated less favorably than another ”
Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification (2017, positive): “Zliobaite provides a comprehensive survey on metrics for measuring disparate impact, 7 which we will summarize here.”
Attributing Fair Decisions with Attention Interventions (2021, background): “This phenomenon is called indirect discrimination (Zliobaite, 2015; Hajian and Domingo-Ferrer, 2013; Zhang et al., 2017).”
SemEval-2018 Task 1: Affect in Tweets (2018, background): “Discrimination-aware data mining focuses on measuring discrimination in data (Zliobaite, 2015; Pedreshi et al., 2008; Hajian and Domingo-Ferrer, 2013).”
Is there Gender bias and stereotype in Portuguese Word Embeddings? (2018, background): “Diﬀerent authors [10][18] show that the removal of bias in models produces a negative impact on the quality of the model.”
Fairness without Harm: Decoupled Classifiers with Preference Guarantees (2019, background): “Such performance disparities are now regularly reported (Angwin et al., 2016; Dastin, 2018), eliciting calls for fairness in machine learning (Crawford, 2013), and prompting the development of technical solutions (Zliobaite, 2015; Barocas et al., 2018; Corbett-Davies & Goel, 2018).”
Ethical Considerations in AI-Based Recruitment (2019, background): “Previous surveys [10, 13-18] and work in fairness in machine learning have not addressed the types of biases that past studies have shown occur in the recruitment process, such as confirmation bias (decisions made in the first few minutes of meeting a candidate) and expectation anchor bias (being in”
Themis-ml: A Fairness-Aware Machine Learning Interface for End-To-End Discrimination Discovery and Mitigation (2017, methodology): “Researchers have developed many discrimination discovery and fairnessaware ML methods (Dwork, Hardt, Pitassi, Reingold, & Zemel, 2012; Kamiran & Calders, 2012; Kamiran, Karim, & Zhang, 2012; Kamishima, Akaho, Asoh, & Sakuma, 2012; Kusner, Loftus, Russell, & Silva, 2017; Zafar, Valera, Gomez Rodrigue”