Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification

By Brian M. D’Alessandro, Cathy O'Neil, Tom LaGatta. 2017

201 citations recorded by the source corpus.

Abstract Recent research has helped to cultivate growing awareness that machine-learning systems fueled by big data can create or exacerbate troubling disparities in society. Much of this research comes from outside of the practicing data science community, leaving its members with little concrete guidance to proactively address these concerns. This article introduces issues of discrimination to the data science community on its own terms. In it, we tour the familiar data-mining process while providing a taxonomy of common practices that have the potential to produce unintended discrimination. We also survey how discrimination is commonly measured, and suggest how familiar development processes can be augmented to mitigate systems' discriminatory potential. We advocate that data scientists should be intentional about modeling and reducing discriminatory outcomes. Without doing so, their efforts will result in perpetuating any systemic discrimination that may exist, but under a misleading veil of data-driven objectivity.

Subfields: Artificial Intelligence, Safety Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In AI Ethics, Fairness & Society

Citations in the corpus

A Survey on Bias and Fairness in Machine Learning (2019, methodology): “Pre-processing techniques try to transform the data so that the underlying discrimination is removed [39].”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “For example, combining petty misdemeanors with violent felonies can affect disparities [34].”
AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias (2018, methodology): “These algorithm categories are known as pre-processing, in-processing, and post-processing, respectively (d’Alessandro et al., 2017).”
Human-AI Collaboration in Data Science (2019, background): “In these ways, data science workers engage deeply with data before data science modeling activities, during model selection [62, 68] and after model refinement, such as when testing for bias [5, 72].”
The relationship between trust in AI and trustworthy machine learning technologies (2019, background): “• Fairness Technologies: technologies focused on detection or prevention of discrimination and bias in different demographics [98, 63, 77, 97, 55, 23, 49, 37, 46, 25, 50, 96, 35].”
Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data (2018, background): “14 Low sample size and underestimation of minority groups are not unique to machine learning or electronic health record data but a common issue in other types of studies, such as randomized clinical trials and genetic studies.”
Bias Mitigation Post-processing for Individual and Group Fairness (2018, background): “Machine learning pipelines contain three possible points of intervention to mitigate unwanted bias: the training data, the learning procedure, and the output predictions, with three corresponding classes of bias mitigation algorithms: pre-processing, in-processing, and post-processing [13].”