Keywords
electronic health records - EHR systems - emergency medicine - interactions - information technology - iusability
Background and Significance
Background and Significance
The usability and safety of electronic health records (EHRs) for healthcare delivery remains a major concern for healthcare providers.[1]
[2]
[3]
[4]
[5] Clinical workflow has been defined as the flow of care-related tasks including structuring of clinical tasks, coordinating task performance, enabling the flow of information to support task performance, and monitoring.[6]
[7] Many clinicians feel that the workflows imposed by EHRs are burdensome, require too many clicks, and require the entry of redundant information.[8]
[9] The increased demands on physicians to electronically document care requires significantly more time on the computer to accomplish tasks, which, in large part, is attributable to EHR usability challenges and difficult workflows.[10] The additional cognitive demands and the required time may lead to patient safety events such as dosing errors, failure to detect fatal illnesses, and delays in treatment.[11]
[12]
Several studies have examined workflow, information flow, and work processes in different clinical settings and have identified guidelines to improve workflow processes.[13]
[14]
[15] Ratwani et al showed that there is variability in time to complete EHR tasks, and the accuracy of completing these tasks varies across different EHR products.[16] However, it remains unknown if the variability of workflow patterns explains the variation in task completion time and accuracy. Understanding how specific workflow variances affect time on task and accuracy would allow EHR designers and developers to optimize workflow and standardize the delivery of high-quality care.
The objectives of this study are as follows:
-
To identify variation in workflow patterns for one specific task: emergency physicians placing a magnetic resonance imaging (MRI) order in two different major EHR vendor products across four different provider sites to identify workflow variations.
-
To determine if specific workflow patterns were associated with task completion time and task accuracy.
Methods
Study Design and Participants
A cross-sectional usability study was conducted at four healthcare systems with two major vendor products, Epic Systems and Cerner, each at two different sites.[16] Participants were recruited by an on-site coordinator from each site. Inclusion criteria consisted of clinical use of the EHR system at least two times per week. All participants were compensated, and this study was approved by the Institutional Review Board.
Test Scenario
A single clinical test case for ordering an MRI was developed by emergency medicine physicians and human factors experts. The clinical case included an introduction, initial actions and evaluation, follow-up actions, and disposition. The case was validated by a clinical expert at each site. Placing an MRI order is a common task occurring in the emergency department (ED) that provides insight into EHR design and standards across sites. For the purposes of this study, we focused on the workflow patterns associated with placing MRI order.
The use case was as follows: A 37-year-old presented to the ED complaining of severe midline lower back pain, denies previous back pain and/or trauma, denies history of smoking, alcohol, and illicit drug use, and has bilateral leg weakness and numbness. Participants were then asked to order MRI of the spine (cervical, thoracic, and lumbar) for cord compression. Cord compression represents a potentially life-threatening abnormality with the spinal cord. Because management for this diagnosis is time-sensitive (i.e., the faster it is diagnosed and addressed, the better the outcomes), adding an indication of cord compression to the MRI order can help the radiologist prioritize the study and communicate more quickly with the care team.
Data Collection and Analysis
Testing was conducted in a quiet room, on-site, and used the EHR training environment that closely mirrored the current production environment. At each site, the same fictitious patient data were preloaded into the ED database to produce a realistic EHR environment and minimize variation between sites. The patient profiles included demographic information, medication history, vital signs, and medical history. The participant's interaction with the EHR was recorded using the Morae recording software. The software recorded screen capture data, mouse movements, and keystroke actions. Participants' demographic data including years of clinical experience and EHR use were collected using a questionnaire.
The screen capture videos were examined for each MRI order. The data for nine participants (two participants from site 1A, three participants from site 2A, one participant from site 3B, and three participants from site 4B) could not be analyzed due to data capture failure. To map clinical workflow, two human factor experts conducted a detailed task analysis for the workflow pathway used by each physician. Workflow patterns were evaluated using objective performance metrics, including time to completion and error rates. Task time (time to completion) and task completion (accuracy of order) were recorded.
An emergency medicine physician categorized the participants' MRI orders into the following four categories to describe the accuracy of the order: (1) “success” if the participant completed the correct order with the indication of cord compression; (2) “work around/usability” if the participant completed the correct order with the indication of cord compression but had to use a workaround such as writing out instructions in a field that may be missed by the radiologist; (3) “clinically equivalent” if the participant completed the correct order with an indication of back pain or an indication other than cord compression; and (4) “fail” if the participant failed to order all of the requested studies (cervical, thoracic, and lumbar MRI) or placed a wrong order (e.g., a different test or imaging modality). A consensus-based approach was applied to determine workflow sequence and judgment of accuracy across experts. Statistical analyses were conducted using “rcompanion_1.13.2,” ”vcd_1.4–4,” ”lsr_0.5” in R 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria).[17]
Results
A total of 55 emergency medicine physicians, including attendings and residents, participated in the study. Participant demographics are shown in [Table 1].
Table 1
Participant demographics by site (55 participants)
|
Site 1A (N = 14)
|
Site 2A (N = 15)
|
Site 3B (N = 14)
|
Site 4B (N = 12)
|
|
Role
|
Count
|
Count
|
Count
|
Count
|
|
Attending
|
5
|
15
|
8
|
7
|
|
Resident
|
9
|
0
|
6
|
5
|
|
Experience in role (years)
|
|
1–5
|
12
|
5
|
11
|
7
|
|
6–10
|
1
|
9
|
1
|
1
|
|
11+
|
1
|
1
|
2
|
4
|
|
Experience with EHR (years)
|
|
Mean (standard deviation)
|
2.89 (1.49)
|
6.8 (5.09)
|
3.4 (1.05)
|
3.12 (1.61)
|
Abbreviation: EHR, electronic health record.
Mapping Clinical Workflow
Analysis of the physician's interaction with the EHR indicated that physicians follow different workflows to complete an MRI order ([Fig. 1]). The task required (1) placing orders for spinal segment MRIs and (2) modifying each order with an indication. This resulted in the following four workflow patterns:
-
“Six-step sequential”: first, the user places one order for either cervical, thoracic, or lumbar MRI. Second, the user modifies that single order. Next, the user places an order for a second spinal segment MRI and then modifies that second order. Finally, the user places an order for the third spinal segment MRI and modifies the final order. This workflow includes six steps.
-
“Six-step serial”: first, the user places three orders, one for each spinal segment MRI (cervical, thoracic, and lumbar). Next, the user modifies each of the three orders separately. This workflow includes six steps and multiple task switches.
-
“Four-step batch”: first, the user places three orders, one for each spinal segment MRI (cervical, thoracic, and lumbar). Next, the user modifies all three orders in a single modification (batch modification). This workflow includes four steps.
-
“Two-step batch”: first, the user employs functionality built into the system to place orders for all three spinal segments in a single step (batch order). Next, the user modifies all three orders in a single modification (batch modification). This workflow includes two steps.
Fig. 1 Workflow patterns based on the differences in the way participants select or modify parts of MRI test order. ED, emergency department; MRI, magnetic resonance imaging.
Variability in Clinical Workflow
All sites differed in the workflow options used by physicians to complete the same task ([Fig. 2]). There was a significant difference in the workflow options adopted by physicians at different sites to complete the MRI order (p = 0.002; Fisher's exact test). We observed that only physicians at site 1A used the option to modify all orders together (“four-step batch”). Only physicians at site 3B used the option to order and modify all three MRI tests together (“two-step batch”). Physicians at sites 2A and 4B used workflow option “six-step sequential” or “six-step serial.”
Fig. 2 Frequency of workflow usage across four health care systems.
Accuracy and Workflow
The distribution of clinical correctness (accuracy) varied across workflow options. [Fig. 3] shows the counts of correctness using each workflow option. We found a moderate-to-high interaction between the workflow option used and the correctness of the order (Cramer's V = 0.412; p = 0.002; Fisher's exact test). [Table 2] shows pairwise comparisons of workflow option and clinical correctness of the order, demonstrating statistically significant differences in the clinical correctness achieved by physicians who employed each workflow option. The “two-step batch” approach led to significantly higher clinical correctness of the order compared with all other workflow options.
Fig. 3 Frequency of clinical performance results based on workflow pattern.
Table 2
Posthoc workflow: clinical correctness comparison using Fisher's exact test
|
Comparison
|
p-Value
|
Adjusted p-value
|
|
Two-step batch: four-step batch
|
0.006
|
0.017
|
|
Two-Step Batch: six-step serial
|
0.004
|
0.017
|
|
Two-Step Batch: six-step sequential
|
0.009
|
0.019
|
|
Four-step batch: six-step serial
|
0.812
|
0.974
|
|
Four-step batch: six-step sequential
|
0.702
|
0.974
|
|
Six-step serial: six-step sequential
|
1.000
|
1.000
|
Note: Significant p-Values are depicted in bold.
Time and Workflow
[Fig. 4] shows the mean and standard deviation (SD) of task time using different workflow options. Workflow option “two-step batch” took the shortest time to complete the order (mean = 29.65 seconds; SD = 13.78). One-way analysis of variance (ANOVA) showed significant differences in mean task time between workflow options (p < 0.001). Posthoc analysis using Tukey's Honest Significant Difference showed significant pairwise differences ([Table 3]).
Fig. 4 Workflow and time.
Table 3
Tukey's Honest Significant Difference test: pairwise differences in task time between workflow options
|
Workflow
|
Estimated mean difference in time and confidence interval
|
p-Value
|
|
Four-step batch: two-step batch
|
43.21 (11.66–74.76)
|
<0.001
|
|
Six-step serial: two-step batch
|
57.84 (32.15–83.54)
|
< 0.001
|
|
Six-step sequential: two-step batch
|
27.35 (0.27–54.43)
|
0.047
|
|
Six-step serial: four-step batch
|
14.64(–15.73 to 45.00)
|
0.575
|
|
Six-step sequential: four-step batch
|
–15.86 (–47.41 to 15.69)
|
0.541
|
|
Six-step sequential: six-step serial
|
–30.49 (–56.19 to –4.80)
|
0.014
|
Note: Significant p-Values are depicted in bold.
Clicks and Workflow
[Fig. 5] shows the mean and SD of the number of clicks using each workflow options. Workflow option “two-step batch” took the lowest number of click equivalents (clicks, tabs, wheel scroll, enter) to complete the order (mean = 13.50; SD = 18.87), whereas “six-step serial” took the maximum number of clicks equivalents (mean = 34.60; SD = 14.45). One-way ANOVA showed significant differences in mean task time between workflow options (p = 0.002).
Fig. 5 Workflow and clicks.
Discussion
In this study, we found that the time required to complete a standardized ordering task and the clinical accuracy of that order varied based on the EHR workflow used by physicians. Furthermore, affordances built into different EHR systems influenced physicians' choice of workflow. While physicians who used the “two-step batch” workflow spent less time on orders, required fewer clicks, and had improved accuracy, only one of the four sites had designed their EHR to enable this workflow. In contrast, at two sites, the only possible workflows required six steps, leading to increased time on task and clicks, as well as reduced accuracy. Ultimately, these results demonstrate that EHR design can impact the time to complete tasks and their accuracy, which can affect patient safety as well as clinician frustration and burnout.
For each site, only some of the four workflow options were used. These may be determined by changes made during product implementation at a particular site and further local customizations and modifications. The different workflows identified in placing an MRI order suggest that a physician who works at multiple sites with different EHRs or instances of the same EHR would likely have to adapt to different workflows for doing the same task. The introduction of different workflows for the same task may contribute to increased cognitive load, provider frustration, lack of awareness of system capability or system state, and the inability to optimize task sequence or workflow. Successful EHR implementation requires that end users understand each task-specific workflow (e.g., MRI order), all technology components work properly with the corresponding workflow, and each end user knows what options are available through relevant software components. While local customizations are often needed to support specific health system workflows, usability testing is rarely conducted to evaluate workflow implications. The application of human factor engineering and user-centered design are just two approaches that could help health care systems reduce inefficiencies and improve patient safety.
This study has a few limitations. We evaluated EHR interactions of only a limited number of physicians. We may have missed other potential workflow pathways due to our limited sample size. The study was conducted in an office setting rather than a clinical environment, which may impact fidelity of the study with regard to added stress and other compounding factors common in the clinical environment, such as interruptions. The study considered only MRI orders, and the findings may not extrapolate to other ordering tasks. However, many other orders also have variations in workflows similar to MRI, where some options involve more steps and task switches that can negatively impact care. Moreover, we did not include data on potential confounders such as the organizational factors, EHR training differences, or other contextual factors that may have influenced performance metrics. Nonetheless, given the large effect sizes found in this study, we feel that the workflow chosen is likely the primary determinant of time on task and clinical correctness. Finally, isolated usability tests for a single task within a larger workflow cannot speak of how the variations in that single order impact the overall workflow of patient care.
The introduction of EHRs has overall demonstrated reduced errors, improved patient safety, and supported better patient outcomes, but it has also given rise to new, unanticipated safety challenges.[18]
[19]
[20] Researchers have identified similar issues in the clinical workflow challenges outlined in our study as “goodness-of-fit” issues in the user-friendliness of EHR systems.[9]
[21] Several high-profile reports have called for multidisciplinary approaches to reduce health information technology (IT) related and EHR-related harm to patients. This may be accomplished by establishing a risk-based regulatory framework, developing health IT safety collaboratives including vendors, leveraging policy regulations such as HITECH (Health Information Technology for Economic and Clinical Health) Act to incentivize health IT use, and a general focus on usability as a critical component of safe and effective use of health IT.
Conclusion
This study demonstrates significant variability in the clinical workflow that contributes to inaccurate orders as well as more time and clicks required. By optimizing usability and workflow patterns, the system can better support physicians, leading to improved accuracy and reduced task completion time, which, in turn, can reduce clinician frustration and improve patient safety. The variability highlights the critical impact of the local implementation of an EHR on the ultimate usability and safety of each EHR installation. This research demonstrates that further work is needed to optimize EHRs based on workflow and usability.
Clinical Relevance Statement
Clinical Relevance Statement
EHR ordering systems have a wide variation in workflow options for physicians to place orders, which impacts both time and safety. Our research highlights the impact of different workflow options, namely time to task completion and accuracy. Identifying and standardizing workflows within the EHR can improve and optimize patient care and reduce inefficiencies. This study represents a larger goal to design technology to fit provider workflow rather than training individuals to adapt to poorly designed technology.
