Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis

By Agostina J. Larrazabal, Nicolás Nieto, Victoria Peterson, Diego H. Milone, Enzo Ferrante. Proceedings of the National Academy of Sciences of the United States of America, 2020

627 citations recorded by the source corpus.

Artificial intelligence (AI) systems for computer-aided diagnosis and image-based screening are being adopted worldwide by medical institutions. In such a context, generating fair and unbiased classifiers becomes of paramount importance. The research community of medical image computing is making great efforts in developing more accurate algorithms to assist medical doctors in the difficult task of disease diagnosis. However, little attention is paid to the way databases are collected and how this may influence the performance of AI systems. Our study sheds light on the importance of gender balance in medical imaging datasets used to train AI systems for computer-assisted diagnosis. We provide empirical evidence supported by a large-scale study, based on three deep neural network architectures and two well-known publicly available X-ray image datasets used to diagnose various thoracic diseases under different gender imbalance conditions. We found a consistent decrease in performance for underrepresented genders when a minimum balance is not fulfilled. This raises the alarm for national agencies in charge of regulating and approving computer-assisted diagnosis systems, which should include explicit gender balance and diversity recommendations. We also establish an open problem for the academic medical image computing community which needs to be addressed by novel algorithms endowed with robustness to gender imbalance.

Subfields: Artificial Intelligence, Health Informatics, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/85efad9b06541cf19f095efd177bf849856755b3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Pranav Rajpurkar

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

AI in health and medicine (2022, background): “AI can make healthcare more accessible to under-served groups, but it also risks reinforcing existing inequities, because AI models can perpetuate biases lurking in the data 112 .”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “Several previous investigations have found relatively small disparities in algorithmic performance across demographic groups (Chen et al., 2019; Larrazabal et al., 2020); Seyyed-Kalantari et al. (2020) finds larger disparities in true positive rates across demographic groups, and future work should…”
Artificial intelligence in histopathology: enhancing cancer research and clinical oncology (2022, background): “This is problematic because it can propagate sexism, racism and other types of discrimination inherent in many medical datasets 131,132 .”
Self-supervised learning in medicine and healthcare (2022, background): “In addition, gender-biased datasets yield models that perform better on the majority class 59 .”
Towards Measuring Fairness in AI: The Casual Conversations Dataset (2021, background): “Research has clearly shown that deep networks that achieve a high performance on certain datasets are likely to favor only sub-groups of people due to the imbalanced distribution of the categories in the data [15].”
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation (2022, background): “1 aimed to address is that underrepresentation can be a problem if the minority group is differently predictive from the majority group [85], but does not contain sufficient samples to train a robust model on.”
Transformative Potential of AI in Healthcare: Definitions, Applications, and Navigating the Ethical Landscape and Public Perspectives (2024, background): “For instance, a study by Larrazabal et al. revealed that an imbalanced dataset with unequal representation of males and females led to signiﬁcantly lower performance averages across all diseases within the minority group, suggesting the presence of gender bias [131].”
Disparities in Health Care and the Digital Divide (2021, background): “Agostina and colleagues make the case that databases that are skewed toward one gender significantly over another are worse at diagnosing disease in the under represented population [22 (cid:129) ].”
Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content? (2022, background): “…for language models, for instance, models producing stereotypical and derogatory content [Bender et al. 2021], and for vision model respectively CV datasets highlighting, e.g., gender and racial biases [Denton et al. 2021; Larrazabal et al. 2020; Steed and Caliskan 2021; Wang et al. 2020].”
Artificial intelligence and machine learning for medical imaging: A technology review. (2021, neutral): “Some examples of these issues include gender imbalance [229], racial bias [230], or data heterogeneity due to changes in treat-ment protocols over time [225].”
Surgical data science – from concepts toward clinical translation (2020, background): “A recent study published in the context of radiological data science (Larrazabal et al., 2020), for example, showed that the performance of AI algorithms for a specific sex (e.”