Biases in electronic health record data due to processes within the healthcare system: retrospective observational study

By Denis Agniel, Isaac S. Kohane, Griffin M. Weber. 2018

378 citations recorded by the source corpus.

Abstract Objective To evaluate on a large scale, across 272 common types of laboratory tests, the impact of healthcare processes on the predictive value of electronic health record (EHR) data. Design Retrospective observational study. Setting Two large hospitals in Boston, Massachusetts, with inpatient, emergency, and ambulatory care. Participants All 669 452 patients treated at the two hospitals over one year between 2005 and 2006. Main outcome measures The relative predictive accuracy of each laboratory test for three year survival, using the time of the day, day of the week, and ordering frequency of the test, compared to the value of the test result. Results The presence of a laboratory test order, regardless of any other information about the test result, has a significant association (P<0.001) with the odds of survival in 233 of 272 (86%) tests. Data about the timing of when laboratory tests were ordered were more accurate than the test results in predicting survival in 118 of 174 tests (68%). Conclusions Healthcare processes must be addressed and accounted for in analysis of observational health data. Without careful consideration to context, EHR data are unsuitable for many research questions. However, if explicitly modeled, the same processes that make EHR data complex can be leveraged to gain insight into patients’ state of health.

Subfields: Artificial Intelligence, Epidemiology, Health Information Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Epidemiology

In Health Information Management

Citations in the corpus

Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions (2020, background): “Due to the strongly missing-not-at-random nature of clinical time series, there is often considerable signal in knowing that certain types of measurements were recently taken, irrespective of the measurement values (Agniel et al., 2018).”
Hidden stratification causes clinically meaningful failures in machine learning for medical imaging (2019, background): “. Non-causal confounding features such as healthcare process quantities can also contribute substantially to high model performance on data subsets heavily associated with these confounding variables [35, 2, 1, 40]. Instead of analyzing subsets deﬁned a priori, Mahajan et al. [21] describe algorithm”
The future landscape of large language models in medicine (2023, background): “Still, LLMs have the inherent limitation of reproducing existing medical biases 31 (Supplementary Data, example 2) and perpetuating inequalities related to factors such as race, gender, sexual orientation, and socioeconomic status 30,32 .”
Ethical Machine Learning in Healthcare (2020, background): “, economic incentives may alter diagnosis code logging (81), clinical protocol affects the frequency and observation of abnormal tests (62), historical racial mistrust may delay care and affect patient outcomes (82), and naive data collection can yield inconsistent labels in chest X-rays (56).”
Axes of a revolution: challenges and promises of big data in healthcare (2020, background): “These confounders may lead to high predictive power of a model that is driven by a variety of health processes rather than a true physiological signal 80 .”