Why Unbiased Computational Processes Can Lead to Discriminative Decision Procedures

By Toon Calders, Indrė Žliobaitė. 2013

161 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In AI Ethics, Fairness & Society

Citations in the corpus

On formalizing fairness in prediction with machine learning (2017, background): “Furthermore, some of the assumptions made during the construction of a predictor might not hold in real-life scenarios (8) which leads to discrimination even while satisfying this measure.”
A survey on measuring indirect discrimination in machine learning (2015, background): “Historical data may be biased, incomplete, or record past discriminatory decisions that can easily be transferred to predictive models, and reinforced in new decision making [Calders and Zliobaite 2013].”
On the Fairness of Disentangled Representations (2019, background): “For example, the inherent assumptions of the algorithm might cause discrimination towards protected groups, the data collection process might be biased or the causal graph itself might allow for unfairness because society is unfair [5, 11, 68, 73, 75, 83].”
Blind Justice: Fairness with Encrypted Sensitive Attributes (2018, background): “…make or support important decisions affecting individuals— such as car insurance pricing, r´esum´e ﬁltering or recidivism prediction—might illegally or unfairly discriminate against certain subgroups of the population (Schreurs et al., 2008; Calders & ˇZliobait˙e, 2012; Barocas & Selbst, 2016).”
The Tyranny of Data? The Bright and Dark Sides of Data-Driven Decision-Making for Social Good (2016, background): “Third, algorithms can lead to discrimination as a result of the misuse of certain models in diﬀerent contexts [14].”
A multidisciplinary survey on discrimination analysis (2013, background): “Little, 2001), predictive statistics (Pope and Sydnor, 2007), and data mining (Ruggieri et al., 2010b; Calders and Z̆liobaityė, 2013).”
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation (2017, background): “In these ﬁelds, eﬀorts to understand and mitigate illegal bias or general unfairness in areas from loan acceptance to word embeddings have propagated [4, 7, 8, 16].”
Bias in data‐driven artificial intelligence systems—An introductory survey (2020, background): “If the mis-represented groups coincide with social groups against which there already exists social bias such as prejudice or discrimination, even “unbiased computational processes can lead to discriminative decision procedures” (Calders & Zliobaite, 2013).”
Optimized Data Pre-Processing for Discrimination Prevention (2017, methodology): “Supervised learning algorithms, increasingly used for decision making in applications of consequence, may at ﬁrst be presumed to be fair and devoid of inherent bias, but in fact, inherit any bias or discrimination present in the data on which they are trained (Calders & ˇZliobait˙e, 2013).”
Fairness in Machine Learning: A Survey (2020, background): “Weights can be introduced for multiple purposes: (1) to indicate a frequency count for an instance type (e.g., Reference [52]), (2) to place lower/higher importance on “sensitive” training samples (e.g., References [53, 152, 165]), or (3) to improve classifier stability (e.g., Reference [181]).”
Optimized Pre-Processing for Discrimination Prevention (2017, methodology): “Supervised learning algorithms, increasingly used for decision making in applications of consequence, may at first be presumed to be fair and devoid of inherent bias, but in fact, inherit any bias or discrimination present in the data on which they are trained [Calders and Žliobaitė, 2013].”
A Framework for Understanding Unintended Consequences of Machine Learning (2019, background): “Others have focused on a particular subset of the ML pipeline: Calders and ˇZliobait ˙ e (2013) present assumptions that are commonly made in the modeling process, and problems that arise because of incorrect labels, skewed sampling procedures, or incomplete data.”
Measuring discrimination in algorithmic decision making (2017, background): “Yet decision making by predictive models may discriminate against people, even if the computing process is fair and well-intentioned (Barocas and Selbst 2016; Calders and Zliobaite 2013; Cit-ron and Pasqualle 2014).”
Using sensitive personal data may be necessary for avoiding discrimination in data-driven decision models (2016, background): “This is because most data mining methods assume that modeling data is correct, and represents the population well, which is often not true in reality (Calders and Zliobaite 2013).”