Coarse race data conceals disparities in clinical risk score performance
By Rajiv Movva, Divya Shanmugam, Kaihua Hou, Priya Pathak, John V. Guttag, Nikhil Garg, Emma Pierson. 2023
36 citations recorded by the source corpus.
Healthcare data in the United States often records only a patient's coarse race group: for example, both Indian and Chinese patients are typically coded as"Asian."It is unknown, however, whether this coarse coding conceals meaningful disparities in the performance of clinical risk scores across granular race groups. Here we show that it does. Using data from 418K emergency department visits, we assess clinical risk score performance disparities across 26 granular groups for three outcomes, five risk scores, and four performance metrics. Across outcomes and metrics, we show that the risk scores exhibit significant granular performance disparities within coarse race groups. In fact, variation in performance within coarse groups often *exceeds* the variation between coarse groups. We explore why these disparities arise, finding that outcome rates, feature distributions, and the relationships between features and outcomes all vary significantly across granular groups. Our results suggest that healthcare providers, hospital systems, and machine learning researchers should strive to collect, release, and use granular race data in place of coarse race data, and that existing analyses may significantly underestimate racial disparities in performance.
Subfields: Clinical Psychology, Emergency Medicine, Oncology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Clinical Psychology
In Emergency Medicine
In Oncology
Citations in the corpus
- PaLM 2 Technical Report (2023, background): “We also note the challenges in design choices related to encoding and interpreting references to identity from automated systems (Hanna et al., 2020; Movva et al., 2023), and highlight this in the results related to "japanese people" (24%), "indian people" (19%) and "asian people" (6%).”