Can AI Help Reduce Disparities in General Medical and Mental Health Care?

By Irene Y. Chen, Peter Szolovits, Marzyeh Ghassemi. 2019

373 citations recorded by the source corpus.

Background As machine learning becomes increasingly common in health care applications, concerns have been raised about bias in these systems' data, algorithms, and recommendations. Simply put, as health care improves for some, it might not improve for all. Methods Two case studies are examined using a machine learning algorithm on unstructured clinical and psychiatric notes to predict intensive care unit (ICU) mortality and 30-day psychiatric readmission with respect to race, gender, and insurance payer type as a proxy for socioeconomic status. Results Clinical note topics and psychiatric note topics were heterogenous with respect to race, gender, and insurance payer type, which reflects known clinical findings. Differences in prediction accuracy and therefore machine bias are shown with respect to gender and insurance type for ICU mortality and with respect to insurance policy for psychiatric 30-day readmission. Conclusions This analysis can provide a framework for assessing and identifying disparate impacts of artificial intelligence in health care.

Subfields: Artificial Intelligence, Epidemiology, Health Informatics, AI Ethics, Fairness & Society, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Epidemiology

In Health Informatics

In AI Ethics, Fairness & Society

In Medical & Health AI

Citations in the corpus

A Survey on Bias and Fairness in Machine Learning (2019, background): “[130] also looks at machine-learning models and data utilized in medical fields, and writes about how artificial intelligence in health care has not impacted all patients equally.”
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence (2020, methodology): “For example, Chen et al. (2019) instantiate this idea of using AI to assess systemic biases in order to reduce disparities in medical care, by studying mortality and 30-day psychiatric readmission with respect to race, gender, and insurance payer type as a proxy for socioeconomic status.”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “Several previous investigations have found relatively small disparities in algorithmic performance across demographic groups (Chen et al., 2019; Larrazabal et al., 2020); Seyyed-Kalantari et al. (2020) finds larger disparities in true positive rates across demographic groups, and future work should…”
Do no harm: a roadmap for responsible machine learning for health care (2019, background): “…Kenneth Jung 11 , Katherine Heller 12,13 , David Kale 14 , Mohammed Saeed 15 , Pilar N. Ossorio 16 , Sonoo Thadaney-Israni 17 and Anna Goldenberg 6,8,18,19,20 * Interest in machine-learning applications within medicine has been growing, but few studies have progressed to deployment in patient…”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, background): “The presence of these subtle biases in medical data decreases model reliability, especially when they are not corrected during model development [180], [181].”
Artificial intelligence in clinical and genomic diagnostics (2019, background): “However, tools that will help to address machine bias are being developed, and careful attention to these issues could not only help to resolve machine bias issues but could eventually lead to diagnostic systems that are free from human bias [95].”
On the Impact of Random Seeds on the Fairness of Clinical Classifiers (2021, methodology): “Following prior work, here we adopt a simple measure: The mean differences in model performance across demographic subgroups (Chen et al., 2019).”
Fairness without Harm: Decoupled Classifiers with Preference Guarantees (2019, background): “…attaining improved accuracy subject to these preference guarantees represents an important direction for future research in fair machine learning given growing calls for such methods in medicine (see e.g., Ferryman & Pitcan, 2018; Popejoy & Fullerton, 2016; Vayena et al., 2018; Chen et al., 2019).”
Hurtful words: quantifying biases in clinical contextual word embeddings (2020, background): “on healthcare data may exhibit such biases [12, 13, 69], and it is important to take measures that minimize the impact of bias on predictive systems [48, 65].”
Artificial Intelligence for Mental Health and Mental Illnesses: An Overview (2019, background): “…those involved in making decisions about the selection, testing, implementation, and evaluation of AI technologies must be aware of ethical challenges, including biased data (e.g., subjective and expressive nature of clinical text data; linking of mental illnesses to certain ethnicities) [104].”
Ethical Machine Learning in Healthcare (2020, background): “Hence, it is important to measure and address the downstream impact of models though audits for bias and examination of clinical impact (6).”
CheXclusion: Fairness gaps in deep chest X-ray classifiers (2020, background): “In particular, models may exhibit disparities in performance across protected subgroups, and this could lead to different subgroups receiving different treat-ment [7].”
A Review of Challenges and Opportunities in Machine Learning for Health. (2018, background): “Models trained on such data may in turn exhibit unfair performance for some populations if a machine learning practitioner is unaware of this underlying variation; thus, checking models across groups is important 28 .”