On the Impact of Random Seeds on the Fairness of Clinical Classifiers

By Silvio Amir, Jan-Willem van de Meent, Byron C. Wallace. 2021

20 citations recorded by the source corpus.

Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s). We explore the implications of this phenomenon for model fairness across demographic groups in clinical prediction tasks over electronic health records (EHR) in MIMIC-III —— the standard dataset in clinical NLP research. Apparent subgroup performance varies substantially for seeds that yield similar overall performance, although there is no evidence of a trade-off between overall and subgroup performance. However, we also find that the small sample sizes inherent to looking at intersections of minority groups and somewhat rare conditions limit our ability to accurately estimate disparities. Further, we find that jointly optimizing for high overall performance and low disparities does not yield statistically significant improvements. Our results suggest that fairness work using MIMIC-III should carefully account for variations in apparent differences that may arise from stochasticity and small sample sizes.

Subfields: Artificial Intelligence, Health Information Management, Evaluation & Benchmarks, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

In Safety, Robustness & Interpretability

Citations in the corpus

On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “5 As the model has skewed cumulative distribution towards uncertainty for the minority, any fairness metric defined on the output of such a model will also reflect this instability, and thus manifests as fairness variance in existing literature [3, 46, 49].”