A validation of machine learning-based risk scores in the prehospital setting

By Douglas Spangler, Thomas Hermansson, D. Smékal, Hans Blomberg. 2019

58 citations recorded by the source corpus.

Background: The triage of patients in pre-hospital care is a difficult task, and improved risk assessment tools are needed both at the dispatch center and on the ambulance to differentiate between low- and high-risk patients. This study develops and validates a machine learning-based approach to predicting hospital outcomes based on routinely collected prehospital data. Methods: Dispatch, ambulance, and hospital data were collected in one Swedish region from 2016 to 2017. Dispatch center and ambulance records were used to develop gradient boosting models predicting hospital admission, critical care (defined as admission to an intensive care unit or in-hospital mortality), and two-day mortality. Model predictions were used to generate composite risk scores which were compared to National Early Warning System (NEWS) scores and actual dispatched priorities in a similar but prospectively gathered dataset from 2018. Results: A total of 38203 patients were included from 2016-2018. Concordance indexes (or area under the receiver operating characteristics curve) for dispatched priorities ranged from 0.51 to 0.66, while those for NEWS scores ranged from 0.66 to 0.85. Concordance ranged from 0.71 to 0.80 for risk scores based only on dispatch data, and 0.79 to 0.89 for risk scores including ambulance data. Dispatch data-based risk scores consistently outperformed dispatched priorities in predicting hospital outcomes, while models including ambulance data also consistently outperformed NEWS scores. Model performance in the prospective test dataset was similar to that found using cross-validation, and calibration was comparable to that of NEWS scores.

Subfields: Emergency Medicine, Epidemiology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Emergency Medicine

1. Nan Liu
2. Marcus Eng Hock Ong
3. Feng Xie
4. Benjamin Alan Goldstein
5. Bibhas Chakraborty

In Epidemiology

1. Nan Liu
2. Marcus Eng Hock Ong
3. Feng Xie
4. Benjamin Alan Goldstein
5. Bibhas Chakraborty

Citations in the corpus

• AutoScore: A Machine Learning–Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records (2020, background): “Although numerous machine learning models, such as neural networks [41,42] and ensemble learning models [43,44], have been developed to complement traditional regression models, most of them are black boxes that do not explain their predictions in a way that humans can understand.”