Forecasting emergency department overcrowding using the CEDOCS score – predictable, but not actionable?
Journal
International Journal of Medical Informatics
Type
Article
Date Issued
2026-07
Author(s)
Born, Cornelius
;
;
Kämmerling, Lilly
;
Kittelmann, Luca
;
Schmauch, Martin
;
Berger, Alexander
;
Gottwald, Martin
;
Friedhoff, Nicola
;
Dommasch, Michael
;
Krcmar, Helmut
Abstract
Objective
This study aims to develop an explainable machine learning (ML) model to predict emergency department (ED) overcrowding using the CEDOCS score and evaluate its perceived actionability in clinical practice.
Methods
We retrospectively analyzed all 300,000 visits to two German EDs between 2019 and 2023. We derived electronic health records (EHR), calendar and ambulance features to predict the CEDOCS score in the next three hours. We used SHAP values to analyze feature attributions. In a six-month intervention study, we introduced the ML model at one of the EDs and assessed perceived tool accuracy, actions taken based on the system’s predictions, and stress.
Results
The best-performing models achieved RMSEs of 11.76, 15.92, and 18.34 at Hospital 1 and 16.04, 21.38, and 22.91 at Hospital 2 for the one-, two- and three-hour forecasts, respectively. The current CEDOCS score, time of day, and ambulance diversion by the intensive care units had the strongest correlation with overcrowding. In the intervention study, clinician stress showed a small-to-moderate association with CEDOCS (r = 0.24, p = 0.03). Actions taken were unrelated to CEDOCS (p = 0.18) and model error (p = 0.21) but positively associated with perceived tool accuracy (r = 0.38, p < 0.01).
Conclusion
Our ML model demonstrates good predictive performance in forecasting ED overcrowding using the CEDOCS score. The SHAP analysis aligns with known causes of overcrowding. However, our intervention study suggests that accurate prediction of CEDOCS alone may be insufficient for actionable clinical decision support, as occupancy-based metrics may not fully capture the frontline resource demands that drive clinicians’ operational decisions.
This study aims to develop an explainable machine learning (ML) model to predict emergency department (ED) overcrowding using the CEDOCS score and evaluate its perceived actionability in clinical practice.
Methods
We retrospectively analyzed all 300,000 visits to two German EDs between 2019 and 2023. We derived electronic health records (EHR), calendar and ambulance features to predict the CEDOCS score in the next three hours. We used SHAP values to analyze feature attributions. In a six-month intervention study, we introduced the ML model at one of the EDs and assessed perceived tool accuracy, actions taken based on the system’s predictions, and stress.
Results
The best-performing models achieved RMSEs of 11.76, 15.92, and 18.34 at Hospital 1 and 16.04, 21.38, and 22.91 at Hospital 2 for the one-, two- and three-hour forecasts, respectively. The current CEDOCS score, time of day, and ambulance diversion by the intensive care units had the strongest correlation with overcrowding. In the intervention study, clinician stress showed a small-to-moderate association with CEDOCS (r = 0.24, p = 0.03). Actions taken were unrelated to CEDOCS (p = 0.18) and model error (p = 0.21) but positively associated with perceived tool accuracy (r = 0.38, p < 0.01).
Conclusion
Our ML model demonstrates good predictive performance in forecasting ED overcrowding using the CEDOCS score. The SHAP analysis aligns with known causes of overcrowding. However, our intervention study suggests that accurate prediction of CEDOCS alone may be insufficient for actionable clinical decision support, as occupancy-based metrics may not fully capture the frontline resource demands that drive clinicians’ operational decisions.
HSG Classification
contribution to scientific community
Refereed
Yes
Publisher
Elsevier BV
Subject(s)
Division(s)
File(s)![Thumbnail Image]()
Name
1-s2.0-S1386505626003461-main.pdf
Size
1.21 MB
Format
Adobe PDF
Checksum (MD5)
644e9eaa5c0b45449d8f800bbfdba586