Identifying and Correcting Label Bias in Machine Learning

By Heinrich Jiang, Ofir Nachum. 2019

332 citations recorded by the source corpus.

Datasets often contain biases which unfairly disadvantage certain groups, and classifiers trained on such datasets can inherit these biases. In this paper, we provide a mathematical formulation of how this bias can arise. We do so by assuming the existence of underlying, unknown, and unbiased labels which are overwritten by an agent who intends to provide accurate labels but may have biases against certain groups. Despite the fact that we only observe the biased labels, we are able to show that the bias may nevertheless be corrected by re-weighting the data points without changing the labels. We show, with theoretical guarantees, that training on the re-weighted dataset corresponds to training on the unobserved but unbiased labels, thus leading to an unbiased machine learning classifier. Our procedure is fast and robust and can be used with virtually any learning algorithm. We evaluate on a number of standard machine learning fairness datasets and a variety of fairness notions, finding that our method outperforms standard approaches in achieving fair classification.

Subfields: Artificial Intelligence, Safety Research, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In AI Ethics, Fairness & Society

Citations in the corpus

Does Object Recognition Work for Everyone? (2019, methodology): “Having said that, commonly used techniques to increase fairness, such as instance re-weighting [22], analyzing features [1] may help in training image-recognition systems that work for everyone.”
Towards Debiasing Fact Verification Models (2019, background): “…the training objective is re-weighted from the standard =1 L ( x ( i ) , y ( i ) ) to This re-weighting is independent of the model architecture and can be easily added to any objec-tive, similar to Jiang and Nachum (2019) where they learn instance weights to address labeling bias in datasets.”
FairBatch: Batch Selection for Model Fairness (2020, methodology): “There are two popular fairness approaches: (i) pre-processing where training data is debiased (Choi et al., 2020) or re-weighted (Jiang and Nachum, 2020), and (ii) in-processing in which an interested model is retrained via several fairness approaches such as fairness objectives (Zafar et al.,…”
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2020, background): “Similarly to noise in the sensitive attribute, Blum and Stangl [11], Jiang and Nachum [59] show that fairness constraints are guaranteed to improve the properties of a predictor only under appropriate assumptions on the label noise.”
Fairness in Machine Learning: A Survey (2020, unknown): “This occurs as calibration has limited flexibility [141].”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, methodology): “Specifically, we conduct empirical studies of five fairness algorithms [2, 40, 69, 77, 121] across a suite of five tabular datasets derived from the US Census with target variables like income and travel time to work [31].”