On The Impact of Machine Learning Randomness on Group Fairness

By Prakhar Ganesh, Hong Chang, Martin Strobel, Reza Shokri. 2023

43 citations recorded by the source corpus.

Statistical measures for group fairness in machine learning reflect the gap in performance of algorithms across different groups. These measures, however, exhibit a high variance between different training instances, which makes them unreliable for empirical evaluation of fairness. What causes this high variance? We investigate the impact on group fairness of different sources of randomness in training neural networks. We show that the variance in group fairness measures is rooted in the high volatility of the learning process on under-represented groups. Further, we recognize the dominant source of randomness as the stochasticity of data order during training. Based on these findings, we show how one can control group-level accuracy (i.e., model fairness), with high efficiency and negligible impact on the model’s overall performance, by simply changing the data order for a single epoch.

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, background): “…text corpora, words representing “people” are often used in more similar contexts to words representing “men” than to words representing “women,” and Ganesh et al. (2023) demonstrates that a model’s performance on fairness metrics can be highly dependent on how the model trains on data…”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Therefore, the safety of LLMs is crucially influenced by the quality of the training corpora [85, 195].”