Keywords
EHR workflow design - emergency medicine - screening tool - pediatric sepsis
Background and Significance
Background and Significance
Severe sepsis and septic shock continue to cause significant morbidity and mortality
in children worldwide.[1] To that end, the World Health Organization, along with its decision-making body
the World Health Assembly, adopted a resolution on May 24, 2017, to improve the prevention,
diagnosis, and management of sepsis—a crucial step in reducing the global burden of
sepsis.[2] The prevalence of pediatric severe sepsis and septic shock has been increasing;
however, advances are being made, and mortality is decreasing.[3] Recent estimates suggest that there are approximately 4,500 children who die every
year from sepsis in the United States alone.[4] Delayed antimicrobial therapy beyond 3 hours from sepsis recognition is an independent
risk factor for mortality and prolonged organ failure in pediatric sepsis and septic
shock.[5] The most recent clinical practice parameters published by the American College of Critical Care Medicine emphasize early, goal-directed therapy within the first hour of identification,[6] as early detection leads to improved patient outcomes through earlier fluid resuscitation
and antibiotic therapy.[7]
[8] Implementation of sepsis screening tools and bundles to guide treatment has been
shown to decrease time to fluid administration and delivery of antibiotics.[9]
[10]
[11]
[12]
Many hospital systems now employ an electronic health record (EHR) and it follows
that implementation of an electronic sepsis screening tool should be integrated into
patient care. Due to age-specific parameters and the pediatric patient's unique ability
to compensate with relatively stable vital signs, automated EHR sepsis screening is
not as straightforward as in the adult population. Therefore, automated EHR sepsis
screening tools are not readily available for pediatric populations necessitating
the use of manual or paper processes in many institutions. Recently, a large children's
hospital published a prospective application and study of an emergency department
(ED)-based electronic alert to improve recognition of pediatric sepsis. The introduction
of a vital sign-based electronic sepsis screening tool with bedside clinician documentation
did improve the recognition of sepsis in their pediatric population.[13] Despite the improvement reported in their study, the EHR-based tool relied on a
two-stage alerting system that required additional clinician documentation beyond
that necessary for routine patient care.
Our institution is a large, urban, freestanding, academic quaternary hospital with
nearly 90,000 emergency visits a year. In 2013, our ED started a multidisciplinary
sepsis team that initially focused on timely goal-directed therapy for provider-identified
patients with severe sepsis or septic shock, known as a “sepsis alert.” To allow tracking
of an electronic time zero for sepsis alert activation, a flowsheet row was built
into the EHR (Epic Systems, Corp.) in 2014. In 2015, it was determined that a more
standardized approach to sepsis screening was the next step and the ED sepsis team
adopted the most commonly used septic shock trigger/identification tool ([Fig. 1]) published by the American Academy of Pediatrics Pediatric Sepsis Collaborative.[14] The effectiveness of our interventions were assessed using manual chart review of
patients who had been made a sepsis alert by providers in the ED as well as patients
with admission to the intensive care unit with severe sepsis or septic shock and those
with discharge International Classification of Diseases 10th Revision codes consistent
with severe sepsis or septic shock.[15]
Fig. 1 Septic shock trigger/identification tool.
This trigger tool used a combination of high-risk conditions (defined as malignancy, asplenia including sickle cell disease, bone marrow transplant, central or indwelling line/catheter, solid organ transplant, severe mental retardation/cerebral palsy [limited verbal capacity or nonverbal and partial or complete assistance required
for all activities of daily living], and immunodeficiency/immunocompromised or immunosuppression) along with eight clinical criteria: abnormalities in temperature, capillary refill, pulse and/or skin exam, mental status as well as hypotension, tachycardia, and/or tachypnea. Patients were considered septic if they met 3 of the 8 clinical criteria or had
a high risk condition and met 2 of the 8 criteria. After deployment of this tool,
many children were erroneously identified as at risk for severe sepsis when in fact
they were not clinically concerning for severe sepsis. Subsequently, the tool was
modified to decrease the number of patients erroneously identified as septic. These
changes included increasing the respiratory rate for children > 13 to 24 breaths per
minute, removing mental status abnormalities due to high user variability, and increased
the number of criteria needed to meet the sepsis alert threshold to requiring 4 of
8 clinical criteria or a high-risk condition and 3 of 8 clinical criteria. The paper
tool was utilized for 2 years in the ED. The next step for the sepsis team was to
integrate the manual sepsis screening tool into the EHR.
Objective
The goal of the sepsis team was to integrate the existing sepsis screening tool into
the EHR. The team's goal was to leverage existing documentation in the EHR without
requiring additional documentation by providers. In addition, the tool would result
in automated notification to providers when the criteria threshold of the tool was
met. Here, we will describe the strategy and design method of integrating a manual
sepsis screening process into the EHR and share our results.
Methods
To design an electronic version of the manual screening tool, we brought together
a multidisciplinary team comprised of ED physicians, nurses, clinical informaticists,
and data analysts. With the predominate goal of keeping providers in their routine
clinical workflows without requiring additional documentation, the electronic tool
needed to leverage standard documentation in the EHR compiled by various provider
roles. We developed a process flow map to establish the patterns of clinical tasks
and data collection that occur during the ED patient encounter. Through guided conversation,
our team “walked through” the patient care continuum for which all arrival patterns,
clinical examinations, documentation templates, etc. were reviewed and investigated.
The analysis process identified when (e.g., triage, initial assessment, reassessments, etc.), where (past medical history, home medication history, etc.), and by whom, each criterion of the paper trigger tool was obtained and documented. Some items
to be evaluated by the electronic tool would be historical within the patient's EHR
and not documented in the current encounter such as a high-risk condition.
To determine where a criterion is documented, we created a crosswalk between the paper trigger tool
and current, standard documentation fields in the EHR. For example, abnormal temperature
is discrete and documented as a numeric value in the vital signs temperature field.
After the team completed the crosswalk analysis, all criteria in the paper trigger
tool were found as discrete fields in the current EHR with the exception of historical
temperature abnormality or provider concern for infection. To ensure appropriate inclusion
into the screening tool, a new assessment field was introduced into the intake nursing
workflow which mimics the first question of the paper trigger tool: “Does this patient
present with concern for INFECTION and/or TEMPERATURE abnormality in the last 4 hours?”
([Fig. 2]). We opted to implement this assessment row in the EHR prior to the electronic sepsis
screening tool launch. Compliance with documentation of the assessment was monitored
and feedback was provided to nursing stakeholders in the weeks leading up to the electronic
screening tool deployment. While the possibility of bias may exist as a result of
the introduction of a new assessment field, this question was already present in the
paper screening tool. This information is a required element of sepsis screening and
needed to be included in the electronic tool that was not identified during the crosswalk
analysis.
Fig. 2 Nursing documentation assessment field.
Once satisfied that all the criteria of the paper trigger tool were available in the
EHR, we shifted focus to the timing of EHR documentation. Clinical workflows were
mapped with further emphasis on when and by whom criteria would be documented. For example, nurses document a capillary refill assessment
during the triage exam and again at reassessment. We discovered each criterion would
likely be obtained and documented repeatedly, often at routine intervals; however,
vital signs and skin and perfusion exams were documented with increased frequency
in acutely ill patients and those requiring additional interventions. Essentially,
higher acuity patients are likely to have many of the sepsis criteria documented more
frequently, thus allowing for a more dynamic screening tool.
We validated the performance of the automated electronic sepsis screening tool through
retrospective, EHR data abstraction. We used QlikView, a business intelligence software
provided by Qlik (Version 11.2 SR 12. Release Date 6/9/2015. Radnor, Pennsylvania,
United States), to evaluate the presence of documented sepsis criteria for all ED
encounters at our institution over the 15-month period from January 1, 2016 to March
31, 2017. We focused on ensuring all historical patients documented as a sepsis alert
would have been captured by the electronic sepsis screening tool. By evaluating documented
criteria every 15 minutes throughout an encounter, we were not only able to ascertain
whether a patient would or would not have been captured by the electronic sepsis screening
tool but know the approximate time the screening threshold was met.
Once we validated that our criteria could be successfully identified within the EHR,
the screening tool was configured using a series of rules or conditions to be evaluated
in a scoring logic. Each criterion, such as assessment abnormalities or historical
documentation of malignancy, was assigned a score. The presence of criteria documentation
is then scored within the screening tool. The EHR continuously evaluates each criterion
score until a threshold (positive screen) score is met. The electronic sepsis screening
tool evaluates all patients that arrive to the ED and throughout the entire ED encounter.
Data elements were extracted and scored from the medical history, medication record,
vital signs, and physical assessment.
Once the automatic trigger tool was designed, the team sought to automate provider
notification of patients who met sepsis screening threshold. Ultimately, it was determined
that an interruptive system alert would be used to notify providers when a patient met the screening threshold which
would prompt a sepsis huddle. The tool would include two interruptive system alerts, one to nursing and one to
physicians and nurse practitioners. The alert to nursing provides clinical decision
support to initiate a sepsis huddle, whereas the physician and nurse practitioner
alert displays a list of clinical criteria met and potential management options. The
sepsis huddle is comprised of a nurse and a fellow or attending physician or nurse
practitioner who would examine the patient at bedside and document the outcome as
an intervention of the interruptive system alert. Subsequent management tools such
as standard treatment order sets would be hosted in the electronic workflow.
To test the performance of the electronic sepsis screening tool and timing of provider
notification, the electronic sepsis screening tool, but not the alerts, was deployed
in the live EHR. The provider alerts were not displayed to end users during the 8-week
testing period and the manual, paper process remained unchanged. Data were abstracted
(sepsis screening scores and timing of alerts) from the patient's EHR behind the scenes
from the live environment to assess the effectiveness of the electronic tool. Documentation
of a manual screening sepsis alert or time zero (the current, paper process) was subsequently
compared with the time when the electronic screening tool would have initiated an
interruptive system alert.
Statistical comparison of the electronic sepsis screening tool to the manual process
was completed by a two-tail paired t-test. Statistical analysis was performed by R (R: A language and environment for
statistical computing. R Core Team, Vienna, Austria, 2017) using Stargazer (Stargazer:
Well-formatted regression and summary statistics tables. R Package 5.2, Harvard University,
Cambridge, Massachusetts, United States, 2015).
Results
In an 8-week testing period, the electronic sepsis screening tool identified 100%
of patients flagged by the manual process (n = 29), and did so, on average, 68 minutes earlier. This difference was found to be
statistically significant (p < 0.001, d = 1.52, 95% confidence interval [0.40, 0.84]) ([Table 1]). [Table 2] demonstrates the characteristics of the 29 patients identified as sepsis alerts
in the ED during the 8-week testing period. Sixteen were female (55%) and 13 were
male (45%). Thirteen patients fell between 12 and 18 years of age, with 1 infant (0–3
months) identified, 4 toddlers (1–4 years), 6 school age children (4–12 years), and
5 adults (> 18 years).
Table 1
Comparison of manual and electronic sepsis screening tools
|
Manual documentation
|
BPA
|
|
Alerts
|
29
|
29
|
|
Mean, min to alert
|
127
|
59
|
|
Range, min to alert
|
17–587
|
1–524
|
|
Median, min to alert
|
89
|
15
|
Abbreviations: BPA, best practice advisory; CI, confidence interval.
Note: Paired t-test on log10 of minutes to alert: t(28) = 5.78;p < 0.001; d = 1.52; 95% CI (0.40, 0.84).
Table 2
Sepsis alert patient characteristics (n = 29)
|
Sepsis alert patient characteristics
|
Patients
|
|
Age category
|
|
|
< 1 y
|
1 (3%)
|
|
≥ 1–4 y
|
4 (14%)
|
|
≥ 4–12 y
|
6 (21%)
|
|
≥ 12–18 y
|
13 (45%)
|
|
≥ 18 y
|
5 (17%)
|
|
Gender
|
|
|
Female
|
16 (55%)
|
|
Male
|
13 (45%)
|
|
Sepsis criteria[a]
|
|
|
Tachycardia
|
25 (86%)
|
|
Tachypnea
|
25 (86%)
|
|
Temperature abnormality
|
20 (69%)
|
|
Skin abnormality
|
20 (69%)
|
|
Capillary refill abnormality
|
18 (62%)
|
|
High-risk condition
|
18 (62%)
|
|
Concern for infection/temperature abnormality
|
17 (59%)
|
|
Hypotension
|
15 (52%)
|
|
Pulse abnormality
|
4 (14%)
|
a Patients can meet multiple criteria.
Of the 29 patients, the most common sepsis alert criteria included 25 (86%) patients
with tachycardia and 25 (86%) patients with tachypnea. The next most common sepsis
alert criteria included 20 (69%) patients with a temperature abnormality, 20 (69%)
patients with a skin abnormality, 18 (62%) patients with a documented high-risk condition,
18 (62%) patients with a capillary refill abnormality, and 17(59%) patients who had
a concern for historical temperature abnormality or infection as documented by the
nurse during the intake exam. Hypotension was present in 15 (52%) patients and 4 (14%)
of the 29 patients met pulse abnormality criteria.
Discussion
Sepsis continues to be a leading cause of morbidity and mortality worldwide,[16] despite this, the clinical presentation of sepsis is nonspecific and dependent on
synthesis of varied clinical information making identification of septic patients
challenging. Widely accepted practice has been to identify sepsis through manual application
of paper screening tools. In 2015, our institution adopted a modified version of the
sepsis trigger/identification tool from the Pediatric Septic Shock Collaborative of
the American Academy of Pediatrics. While the work of this early collaborative was vital in the development of a tool
that supports both the identification of patients with signs and symptoms of sepsis
and offers the opportunity to intervene during the “golden hour” of resuscitation,
the utility of the paper trigger tool depends upon routine and repeated application
of the tool and may miss opportunities for early sepsis detection. It also allows
user bias such as the clinician only applying the tool to patients they already suspect
to be septic. Unless reapplied throughout the ED stay, it has the potential to miss
patients who develop sepsis during their encounter. The manual tool only includes
those patients where there is known fever, hypothermia, or concern for infection on
ED presentation.
The increasing and widespread use of EHRs offers a unique way to intervene early in
a patient encounter. On trend with other EDs throughout the nation, we wanted to leverage
the EHR's ability to recognize discrete documentation of sepsis criteria and prompt
notification to ED providers early and reliably, and eliminate user variability and
clinician bias. Additionally, we wanted this process to not incur additional burden
on provider workflow. Previously published attempts at creating electronic sepsis
screening tools were limited due to need for additional documentation, a single assessment
that did not update throughout the encounter,[17]
[18] or a need for reassessment to look for specific subsequent criteria.[19]
[20] Our tool does not introduce or require additional documentation with the exception
of a single question during triage. Since the tool is electronic and automated, it
assesses all patients on arrival to the ED and on instant of documentation throughout
an ED encounter while still allowing providers to continue with their routine clinical
workflows.
The electronic sepsis screening tool was found to be as accurate as a paper, manual
process. Based on the validation phase of our design, the electronic tool identified
100% (n = 29 over an 8-week testing period) of patients flagged by the manual process which
included a wide variety of patients across age groups and medical complexities. In
addition to accurate identification, the electronic tool identified patients 68 minutes
earlier in the ED encounter when compared with the manual process. The American College of Critical Care Medicine emphasizes early, goal-directed therapy within the first hour of identification,[5] as early detection leads to improved patient outcomes through earlier fluid resuscitation
and antibiotic therapy.[6] Our automated, electronic sepsis screening tool supports this goal. This statistically
significant finding in the validation phase of our work may be the result of consistent
and repeated application of the electronic tool independent of the patient's clinical
status. It removes the limitations to identification of sepsis that could previously
have been explained by timing of provider reevaluation or communication of changes
in patient status and ultimately results in timely notification of a concern for sepsis.
In the informatics literature, Embi et al found that providers have decreasing response
rates to EHR-based system alerts over time.[21] Accordingly, we sought to create an alerting system that maintains an appropriate
level of concern without an unwarranted number of alerts. For this reason, we created
a tool that minimizes excessive alerts while maintaining a high level of sensitivity.
By doing this, we maximize the screening capability of the electronic sepsis screening
tool.
Limitations
This work was completed at a single institution, and as such, the generalizability
of these results may be limited by differences in population. However, the population
at our institution typically includes a broad mix of medical and surgical patients
in an ED that routinely sees nearly 90,000 patients per year.
Another potential limitation of this process is ensuring the accuracy of the patient's
medical history as entered into the medical record. Our institution is unique in that
the vast majority of patients with high-risk conditions in the geographic area are
seen at our institution, so the historical medical information is typically preexisting
in the EHR; however, in other geographic settings where the patient may not be primarily
followed at that institution, information may be missing from the record and thus
not used as dynamically to calculate a risk for sepsis.
Finally, this tool is intended as a replacement of a manual sepsis screening tool,
and patients who do not meet the clinical criteria for sepsis may be missed by both
the manual and electronic sepsis screening tools.
Conclusion
The automated, electronic sepsis screening tool is as accurate as a validated manual
process and identified patients at risk for sepsis significantly earlier than the manual process. Deployment of this electronic tool has the capability
to improve timely sepsis detection and management of patients at risk for sepsis without
requiring additional documentation by providers. Importantly, this electronic sepsis
screening tool keeps providers in their routine clinical workflows and does not introduce
bias.
Multiple Choice Questions
Multiple Choice Questions
-
Which of the following methods validated the performance of the new electronic screening
tool?
-
Retrospective manual chart review.
-
EHR data abstraction using business intelligence software.
-
Deployment of the tool in the live environment to end users.
-
Requiring additional sepsis assessments in the provider workflows.
Correct Answer: The correct answer is option b. We used Qlikview, a business intelligence software
provided by Qlik to evaluate the presence of documented sepsis criteria for all ED
encounters at our institution over the 15-month period from January 1, 2016 to March
31, 2017. We focused on ensuring all historical patients documented as a sepsis alert
would have been captured by the electronic sepsis screening tool.
-
Which of the following criteria in the EHR was missing as a discrete data element
during the crosswalk analysis?
-
Vital sign abnormalities.
-
Abnormalities in physical assessment (i.e., capillary refill, pulse abnormality).
-
Patient information related to infection and/or temperature abnormalities.
-
History of high-risk conditions that place the patient at increased risk for sepsis.
Correct Answer: The correct answer is option c. After the team completed the crosswalk analysis,
all criteria in the paper trigger tool were found as discrete fields in the current
EHR with the exception of historical temperature abnormality or provider concern for
infection (see [Fig. 2]).
