Sources of bias in artificial intelligence that perpetuate healthcare disparities—A global review
By Leo Anthony Celi, J. Cellini, Marie Charpignon, E. Dee, Franck Dernoncourt, René Eber, W. Mitchell, Lama Moukheiber, Julian Schirmer, J. Situ and 4 more. PLOS Digital Health, 2022
442 citations recorded by the source corpus.
Background While artificial intelligence (AI) offers possibilities of advanced clinical prediction and decision-making in healthcare, models trained on relatively homogeneous datasets, and populations poorly-representative of underlying diversity, limits generalisability and risks biased AI-based decisions. Here, we describe the landscape of AI in clinical medicine to delineate population and data-source disparities. Methods We performed a scoping review of clinical papers published in PubMed in 2019 using AI techniques. We assessed differences in dataset country source, clinical specialty, and author nationality, sex, and expertise. A manually tagged subsample of PubMed articles was used to train a model, leveraging transfer-learning techniques (building upon an existing BioBERT model) to predict eligibility for inclusion (original, human, clinical AI literature). Of all eligible articles, database country source and clinical specialty were manually labelled. A BioBERT-based model predicted first/last author expertise. Author nationality was determined using corresponding affiliated institution information using Entrez Direct. And first/last author sex was evaluated using the Gendarize.io API. Results Our search yielded 30,576 articles, of which 7,314 (23.9%) were eligible for further analysis. Most databases came from the US (40.8%) and China (13.7%). Radiology was the most represented clinical specialty (40.4%), followed by pathology (9.1%). Authors were primarily from either China (24.0%) or the US (18.4%). First and last authors were predominately data experts (i.e., statisticians) (59.6% and 53.9% respectively) rather than clinicians. And the majority of first/last authors were male (74.1%). Interpretation U.S. and Chinese datasets and authors were disproportionately overrepresented in clinical AI, and almost all of the top 10 databases and author nationalities were from high income countries (HICs). AI techniques were most commonly employed for image-rich specialties, and authors were predominantly male, with non-clinical backgrounds. Development of technological infrastructure in data-poor regions, and diligence in external validation and model re-calibration prior to clinical implementation in the short-term, are crucial in ensuring clinical AI is meaningful for broader populations, and to avoid perpetuating global health inequity.
Subfields: General Health Professions, Health Informatics, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/429018881ad753f0d5be21fdad6c036c44566005
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Health Informatics
In Radiology, Nuclear Medicine and Imaging
Citations in the corpus
- Bias in medical AI: Implications for clinical decision-making (2024, background): “Journals and funding agencies might encourage and incentivize multidisciplinary co-authorship for studies that develop AI solutions intended for clinical use, particularly in underrepresented medical AI domains like primary care, mental health, and pediatrics [40].”
- Bias in artificial intelligence algorithms and recommendations for mitigation (2023, background): “In a recent review of the literature on clinical AI, Celi and colleagues reported that over half of the databases used to train models primarily reflected patients treated in the US and China [55].”
- FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2023, unknown): “For example, the FAIR guideline 10 for data management has been widely adopted by researchers, organisations, and authorities, as the principles provide a structured framework for standardising and enhancing the tasks of data collection, curation, organisation, and storage.”