Dissecting racial bias in an algorithm used to manage the health of populations

By Z. Obermeyer, Brian W. Powers, C. Vogeli, Sendhil Mullainathan. 2019

5,573 citations recorded by the source corpus.

Racial bias in health algorithms The U.S. health care system uses commercial algorithms to guide health decisions. Obermeyer et al. find evidence of racial bias in one widely used algorithm, such that Black patients assigned the same level of risk by the algorithm are sicker than White patients (see the Perspective by Benjamin). The authors estimated that this racial bias reduces the number of Black patients identified for extra care by more than half. Bias occurs because the algorithm uses health costs as a proxy for health needs. Less money is spent on Black patients who have the same level of need, and the algorithm thus falsely concludes that Black patients are healthier than equally sick White patients. Reformulating the algorithm so that it no longer uses costs as a proxy for needs eliminates the racial bias in predicting who needs extra care. Science, this issue p. 447; see also p. 421 A health algorithm that uses health costs as a proxy for health needs leads to racial bias against Black patients. Health systems rely on commercial prediction algorithms to identify and help patients with complex health needs. We show that a widely used algorithm, typical of this industry-wide approach and affecting millions of patients, exhibits significant racial bias: At a given risk score, Black patients are considerably sicker than White patients, as evidenced by signs of uncontrolled illnesses. Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%. The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients. Thus, despite health care cost appearing to be an effective proxy for health by some measures of predictive accuracy, large racial biases arise. We suggest that the choice of convenient, seemingly effective proxies for ground truth can be an important source of algorithmic bias in many contexts.

Subfields: Economics and Econometrics, Finance, General Health Professions, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

  1. Gary S. Collins
  2. Karel G. M. Moons
  3. John V. Guttag
  4. Harini Suresh
  5. John P. A. Ioannidis
  6. Bin Yu
  7. Emilio Ferrara
  8. William S. Isaac

In Finance

  1. F. Kroeger
  2. Iason Gabriel
  3. Bin Yu
  4. Adrian Weller
  5. Prasanna Sattigeri
  6. John V. Guttag
  7. Yoshua Bengio
  8. M. El-Haddad,

In General Health Professions

  1. Gary S. Collins
  2. Karel G. M. Moons
  3. David Moher
  4. John P. A. Ioannidis
  5. Iason Gabriel
  6. Sharad Goel
  7. Eduard H. Hovy
  8. Meredith Ringel Morris

In Safety, Robustness & Interpretability

  1. Emilio Ferrara
  2. Adrian Weller
  3. Vinodkumar Prabhakaran
  4. John V. Guttag
  5. Chandan Singh
  6. Harini Suresh
  7. James Y. Zou
  8. Bin Yu

Citations in the corpus