Multicenter Comparison of Machine Learning Methods and Conventional Regression for Predicting Clinical Deterioration on the Wards

By M. M. P. Matthew M. Churpek, Trevor C. Yuen, Christopher J. Winslow, David O. Meltzer, Michael W. Kattan, Dana P. Edelson. Critical Care Medicine, 2016

564 citations recorded by the source corpus.

Objective:Machine learning methods are flexible prediction algorithms that may be more accurate than conventional regression. We compared the accuracy of different techniques for detecting clinical deterioration on the wards in a large, multicenter database. Design:Observational cohort study. Setting:Five hospitals, from November 2008 until January 2013. Patients:Hospitalized ward patients Interventions:None Measurements And Main Results:Demographic variables, laboratory values, and vital signs were utilized in a discrete-time survival analysis framework to predict the combined outcome of cardiac arrest, intensive care unit transfer, or death. Two logistic regression models (one using linear predictor terms and a second utilizing restricted cubic splines) were compared to several different machine learning methods. The models were derived in the first 60% of the data by date and then validated in the next 40%. For model derivation, each event time window was matched to a non-event window. All models were compared to each other and to the Modified Early Warning score, a commonly cited early warning score, using the area under the receiver operating characteristic curve (AUC). A total of 269,999 patients were admitted, and 424 cardiac arrests, 13,188 intensive care unit transfers, and 2,840 deaths occurred in the study. In the validation dataset, the random forest model was the most accurate model (AUC, 0.80 [95% CI, 0.80-0.80]). The logistic regression model with spline predictors was more accurate than the model utilizing linear predictors (AUC, 0.77 vs 0.74; p < 0.01), and all models were more accurate than the MEWS (AUC, 0.70 [95% CI, 0.70-0.70]). Conclusions:In this multicenter study, we found that several machine learning methods more accurately predicted clinical deterioration than logistic regression. Use of detection algorithms derived from these techniques may result in improved identification of critically ill patients on the wards.

Subfields: Cardiology and Cardiovascular Medicine, Epidemiology, Health Informatics

View Paper: https://www.semanticscholar.org/paper/070af49f0ad24bd1e0e305fd206933aa49eb7107

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cardiology and Cardiovascular Medicine

In Epidemiology

In Health Informatics

Citations in the corpus

Artificial Intelligence in Cardiology. (2018, background): “(12) demonstrated the utility of machine learning feature selection in their paper comparing methods for prediction of clinical deterioration on the wards.”
Comparison of Conventional Statistical Methods with Machine Learning in Medicine: Diagnosis, Drug Development, and Treatment (2020, background): “In particular, ML can better predict clinical deterioration in the ward [26], mortality in acute coronary syndrome [27], survival in patients with epithelial ovarian cancer [28], complications of bariatric surgery [29], and risk of metabolic syndrome [30].”
An Algorithm Based on Deep Learning for Predicting In‐Hospital Cardiac Arrest (2018, methodology): “To show the effect of deep learning, we used fewer variables than a MEWS.”
A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. (2019, background): “[83] Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW, Edelson DP.”
Comparison of Variable Selection Methods for Clinical Predictive Modeling (2018, methodology): “Modern data modeling methods, mostly derived from the machine learning literature, are also becoming widely available and are starting to be used by clinical researchers [2–4].”
Scalable and accurate deep learning with electronic health records (2018, result): “unexpected readmission (0.75-0.76 vs 0.69)[71] and increased length of stay (0.85-0.86 vs 0.77) [62]. Direct comparisons to other studies are diﬃcult [98] because of diﬀerent underlying study designs [52, 29, 97, 103, 77, 21, 80, 89, 53, 14, 84, 31, 104, 87, 96] incomplete deﬁnitions of cohorts and ”