AutoPEWS: Automating Pediatric Early Warning Score Calculation Improves Accuracy Without Sacrificing Predictive Ability (original) (raw)
Related papers
Resuscitation, 2020
Background: Early warning tools have been widely implemented without evidence to guide (a) recognition and (b) response team expertise optimisation. With growing databases from MET-calls and digital hospitals, we now have access to guiding information. The Queensland Adult-Deterioration-Detection-System (Q-ADDS) is widely used and requires validation. Aim: Compare the accuracy of Q-ADDS to National Early Warning Score (NEWS), Between-the-Flags (BTF) and the electronic Cardiac Arrest Risk Triage Score (eCART)). Methods: Data from the Chicago University hospital database were used. Clinical deterioration was defined as unplanned admission to ICU or death. Currently used NEWS, BTF and eCART trigger thresholds were compared with a clinically endorsed Q-ADDS variant. Results: Of 224,912 admissions, 11,706 (5%) experienced clinical deterioration. Q-ADDS (AUC 0.71) and NEWS (AUC 0.72) had similar predictive accuracy, BTF (AUC 0.64) had the lowest, and eCART (AUC 0.76) the highest. Early warning alert (advising ward MO review) had similar NPV (99.2À99.3%), for all the four tools however sensitivity varied (%: Q-ADDS = 47/NEWS = 49/BTF = 66/eCART = 40), as did alerting rate (% vitals sets: Q-ADDS = 1.4/NEWS = 3.5/ BTF = 4.1/eCART = 3.4). MET alert (advising MET/critical-care review) had similar NPV for all the four tools (99.1À99.2%), however sensitivity varied
Validation of computerized automatic calculation of the sequential organ failure assessment score
Critical care research and practice, 2013
Purpose. To validate the use of a computer program for the automatic calculation of the sequential organ failure assessment (SOFA) score, as compared to the gold standard of manual chart review. Materials and Methods. Adult admissions (age > 18 years) to the medical ICU with a length of stay greater than 24 hours were studied in the setting of an academic tertiary referral center. A retrospective cross-sectional analysis was performed using a derivation cohort to compare automatic calculation of the SOFA score to the gold standard of manual chart review. After critical appraisal of sources of disagreement, another analysis was performed using an independent validation cohort. Then, a prospective observational analysis was performed using an implementation of this computer program in AWARE Dashboard, which is an existing real-time patient EMR system for use in the ICU. Results. Good agreement between the manual and automatic SOFA calculations was observed for both the derivation (...
The New American Journal of Medicine, 2021
National Early Warning Score (NEWS) 2 was developed by the Royal College of Physicians to be used as early detection of in-hospital patient deterioration. The score has been shown to improve prognosis and triage efficiency. While the benefit of using a triage system has been established, in the real world, inefficient manual data entry and human error contribute to possible delays in identifying patients requiring intensive care transfers. Our team wanted to create an automated predictive model using informatics to improve the quality of patient care and healthcare provider workflow efficiency to identify high-risk patients who would benefit from early interventions by rapid-response teams.
Using Statistical and Machine Learning Methods to Evaluate the Prognostic Accuracy of SIRS and qSOFA
Healthcare informatics research, 2018
The objective of this study was to compare the performance of two popularly used early sepsis diagnostic criteria, systemic inflammatory response syndrome (SIRS) and quick Sepsis-related Organ Failure Assessment (qSOFA), using statistical and machine learning approaches. This retrospective study examined patient visits in Emergency Department (ED) with sepsis related diagnosis. The outcome was 28-day in-hospital mortality. Using odds ratio (OR) and modeling methods (decision tree [DT], multivariate logistic regression [LR], and naïve Bayes [NB]), the relationships between diagnostic criteria and mortality were examined. Of 132,704 eligible patient visits, 14% died within 28 days of ED admission. The association of qSOFA ≥2 with mortality (OR = 3.06; 95% confidence interval [CI], 2.96-3.17) greater than the association of SIRS ≥2 with mortality (OR = 1.22; 95% CI, 1.18-1.26). The area under the ROC curve for qSOFA (AUROC = 0.70) was significantly greater than for SIRS (AUROC = 0.63)....
Prospective validation of a near real-time EHR-integrated automated SOFA score calculator
International journal of medical informatics, 2017
We created an algorithm for automated Sequential Organ Failure Assessment (SOFA) score calculation within the Electronic Health Record (EHR) to facilitate detection of sepsis based on the Third International Consensus Definitions for Sepsis and Septic Shock (SEPSIS-3) clinical definition. We evaluated the accuracy of near real-time and daily automated SOFA score calculation compared with manual score calculation. Automated SOFA scoring computer programs were developed using available EHR data sources and integrated into a critical care focused patient care dashboard at Mayo Clinic in Rochester, Minnesota. We prospectively compared the accuracy of automated versus manual calculation for a sample of patients admitted to the medical intensive care unit at Mayo Clinic Hospitals in Rochester, Minnesota and Jacksonville, Florida. Agreement was calculated with Cohen's kappa statistic. Reason for discrepancy was tabulated during manual review. Random spot check comparisons were performe...
Early Deterioration Indicator: Data- driven approach to detecting deterioration in general ward
Resuscitation, 2017
Early detection of deterioration could facilitate more timely interventions which are instrumental in reducing transfer to higher levels of care such as Intensive Care Unit (ICU) and mortality.(1,2) METHODS AND RESULTS: We developed the Early Deterioration Indicator (EDI) which uses log likelihood risk of vital signs to calculate continuous risk scores. EDI was developed using data from 11864 general ward admissions. To validate EDI, we calculated EDI scores on an additional 2418 general ward stays and compared it to the Modified Early Warning Score (MEWS) and National Early Warning Score (NEWS). EDI was trained using the most significant variables in predicting deterioration by leveraging the knowledge from a large dataset through data mining. It was implemented electronically for continuous automatic computation. The discriminative performance of EDI, MEWS, and NEWS was calculated before deterioration using the area under the receiver operating characteristic curve (AUROC). Additi...
Pediatric Quality & Safety, 2020
Introduction: Pediatric in-hospital cardiac arrests and emergent transfers to the pediatric intensive care unit (ICU) represent a serious patient safety concern with associated increased morbidity and mortality. Some institutions have turned to the electronic health record and predictive analytics in search of earlier and more accurate detection of patients at risk for decompensation. Methods: Objective electronic health record data from 2011 to 2017 was utilized to develop an automated early warning system score aimed at identifying hospitalized children at risk of clinical deterioration. Five vital sign measurements and supplemental oxygen requirement data were used to build the Vitals Risk Index (VRI) model, using multivariate logistic regression. We compared the VRI to the hospital's existing early warning system, an adaptation of Monaghan's Pediatric Early Warning Score system (PEWS). The patient population included hospitalized children 18 years of age and younger while being cared for outside of the ICU. This dataset included 158 case hospitalizations (102 emergent transfers to the ICU and 56 "code blue" events) and 135,597 control hospitalizations. Results: When identifying deteriorating patients 2 hours before an event, there was no significant difference between Pediatric Early Warning Score and VRI's areas under the receiver operating characteristic curve at false-positive rates ≤ 10% (pAUC 10 of 0.065 and 0.064, respectively; P = 0.74), a threshold chosen to compare the 2 approaches under clinically tolerable false-positive rates. Conclusions: The VRI represents an objective, simple, and automated predictive analytics tool for identifying hospitalized pediatric patients at risk of deteriorating outside of the ICU setting.
Early warning score using electronically-held data for patients discharged from intensive care units
2020
Rationale. Intensive care units (ICUs) admit the most severely ill patients. Once these patients are discharged from the ICU to a step-down ward, they continue to have their vital signs monitored by nursing staff, with early warning score (EWS) systems being used to identify those at risk of deterioration. Objectives. We report the development and validation of an enhanced continuous scoring system for predicting adverse events, which combines vital signs measured routinely on acute care wards (as used by most EWSs) with a risk score of a future adverse event calculated on discharge from ICU. Methods. A modified Delphi process identified common, and candidate variables frequently collected and stored in electronic records as the basis for a ‗static' score of the patient's condition immediately after discharge from the ICU. L1-regularised logistic regression was used to estimate the in-hospital risk of future adverse event. We then constructed a model of physiological normality using vital-sign data from the day of hospital discharge, which is combined with the static score and used continuously to quantify and update the patient's risk of deterioration throughout their hospital stay. Data from two NHS Foundation Trusts (UK) were used to develop and (externally) validate the model. Measurements and Main Results. A total of 12,394 vital-sign measurements were acquired from 273 patients after ICU discharge for the development set, and 4,831 from 136 patients in the validation cohort. Outcome validation of our model yielded an area under the receiver operating characteristic curve (AUROC) of 0.724 for predicting ICU re-admission or in-hospital death within 24h. It showed an improved performance with respect to other competitive risk scoring systems, including the National EWS (NEWS, 0.653). Conclusion. We showed that a scoring system incorporating data from a patient's stay in ICU has better performance than commonly-used EWS systems based on vital signs alone.
International Journal of Environmental Research and Public Health
The objective of this study was to analyze and compare the usefulness of quick sequential organ failure assessment score (qSOFA) and sequential organ failure assessment (SOFA) scores for the detection of early (two-day) mortality in patients transported by emergency medical services (EMSs) to the emergency department (ED) (infectious and non-infectious). We performed a multicentric, prospective and blinded end-point study in adults transported with high priority by ambulance from the scene to the ED with the participation of five hospitals. For each score, the area under the curve (AUC) of the receiver operating characteristic (ROC) curve was calculated. We included 870 patients in the final cohort. The median age was 70 years (IQR 54–81 years), and 338 (38.8%) of the participants were women. Two-day mortality was 8.3% (73 cases), and 20.9% of cases were of an infectious pathology. For two-day mortality, the qSOFA presented an AUC of 0.812 (95% CI: 0.75–0.87; p < 0.001) globally ...
Validation of the Pediatric Early Warning Score to determine patient deterioration from illness
Paediatrica Indonesiana, 2016
Background Patients who enter the emergency room (er) present with a variety of conditions, ranging from mild to critical. As such, it may be hard to determine which patients are in need of intensive care unit treatment. The Pediatric Early Warning Score (PeWS) has been used to identify signs of critical illness in pediatric patients. Objective To validate the PeWS system for assessing signs of critical illness in pediatric patients at Dr. mohammad Hoesin Hospital, Palembang. Methods Subjects were children aged 1 month to 18 years who received treatment in the er and Pediatrics Ward in Dr. Mohammad Hoesin Hospital in March to April 2015. Assessment with PeWS was based on vital sign examinations. Scores ranged from 0 to 9. The PEWS was generally taken twice, first in the er, then after 6 hours in the ward. We obtained the cutoff point, sensitivity, and specificity of PeWS, in terms of need for pediatric intensive care unit (PICu) treatment. Results one hundred fifty patients were included in this study. Patients with PeW score of 5 or greater in the er were relatively more likely to be transferred to the PICU, with a sensitivity of 94.4% and a specificity of 82.5%. The cutoff point obtained from the ROC curve was score 4.5 with AUC 96.7% (95%CI 93.4 to 99.9%; P<0.001). Conclusion A PEWS score of cutoff ≥5 may be used to determine which patients are in critically ill condition requiring treatment in PICu.