Variation in Electronic Health Record Workflow Patterns: A Multisite study

Abstract

Objectives Electronic health records (EHRs) continue to have significant usability challenges in part due to differences in workflow. The objective of this study was to examine workflow pattern variations for one specific task: emergency physicians placing a magnetic resonance imaging (MRI) order.

Methods A between-subjects usability study was conducted using two different major EHR vendor products across four different provider sites (n = 55). A clinical scenario concerning for spinal cord compression was read to participants who then completed an ordering task using a training environment representative of their native EHR. The primary outcome measures were accuracy, time on task, and number of clicks.

Results We identified four different workflows to complete the same order. One workflow required two steps (enabled at one site), one workflow required four steps (enabled at two sites), and two workflows required six steps to complete the task (available at all sites). Of the 12 physicians who employed the two-step workflow, 8 (67%) had the correct order and correct indication, the average time on task was 29.65 (standard deviation [SD] = 13.77), and the mean number of clicks was 13.5 (SD = 18.87). In contrast, for the 43 physicians who employed other workflows, 7 (21%) had the correct order and correct indication, with the average time on task of 73.1 (SD = 30.12) and mean clicks of 27.64 (SD = 13.25) (p < 0.01 for all three comparisons).

Discussion These different approaches were made possible by technical specifications leading to multiple workflow options available to physicians in the EHR environment. EHR design maximizing usability can reduce the work effort and improve the accuracy of physician ordering.

Keywords

electronic health records - EHR systems - emergency medicine - interactions - information technology - iusability

Background and Significance

The usability and safety of electronic health records (EHRs) for healthcare delivery remains a major concern for healthcare providers.[1] [2] [3] [4] [5] Clinical workflow has been defined as the flow of care-related tasks including structuring of clinical tasks, coordinating task performance, enabling the flow of information to support task performance, and monitoring.[6] [7] Many clinicians feel that the workflows imposed by EHRs are burdensome, require too many clicks, and require the entry of redundant information.[8] [9] The increased demands on physicians to electronically document care requires significantly more time on the computer to accomplish tasks, which, in large part, is attributable to EHR usability challenges and difficult workflows.[10] The additional cognitive demands and the required time may lead to patient safety events such as dosing errors, failure to detect fatal illnesses, and delays in treatment.[11] [12]

Several studies have examined workflow, information flow, and work processes in different clinical settings and have identified guidelines to improve workflow processes.[13] [14] [15] Ratwani et al showed that there is variability in time to complete EHR tasks, and the accuracy of completing these tasks varies across different EHR products.[16] However, it remains unknown if the variability of workflow patterns explains the variation in task completion time and accuracy. Understanding how specific workflow variances affect time on task and accuracy would allow EHR designers and developers to optimize workflow and standardize the delivery of high-quality care.

The objectives of this study are as follows:

• To identify variation in workflow patterns for one specific task: emergency physicians placing a magnetic resonance imaging (MRI) order in two different major EHR vendor products across four different provider sites to identify workflow variations.

• To determine if specific workflow patterns were associated with task completion time and task accuracy.

Methods

Study Design and Participants

A cross-sectional usability study was conducted at four healthcare systems with two major vendor products, Epic Systems and Cerner, each at two different sites.[16] Participants were recruited by an on-site coordinator from each site. Inclusion criteria consisted of clinical use of the EHR system at least two times per week. All participants were compensated, and this study was approved by the Institutional Review Board.

Test Scenario

A single clinical test case for ordering an MRI was developed by emergency medicine physicians and human factors experts. The clinical case included an introduction, initial actions and evaluation, follow-up actions, and disposition. The case was validated by a clinical expert at each site. Placing an MRI order is a common task occurring in the emergency department (ED) that provides insight into EHR design and standards across sites. For the purposes of this study, we focused on the workflow patterns associated with placing MRI order.

The use case was as follows: A 37-year-old presented to the ED complaining of severe midline lower back pain, denies previous back pain and/or trauma, denies history of smoking, alcohol, and illicit drug use, and has bilateral leg weakness and numbness. Participants were then asked to order MRI of the spine (cervical, thoracic, and lumbar) for cord compression. Cord compression represents a potentially life-threatening abnormality with the spinal cord. Because management for this diagnosis is time-sensitive (i.e., the faster it is diagnosed and addressed, the better the outcomes), adding an indication of cord compression to the MRI order can help the radiologist prioritize the study and communicate more quickly with the care team.

Data Collection and Analysis

Testing was conducted in a quiet room, on-site, and used the EHR training environment that closely mirrored the current production environment. At each site, the same fictitious patient data were preloaded into the ED database to produce a realistic EHR environment and minimize variation between sites. The patient profiles included demographic information, medication history, vital signs, and medical history. The participant's interaction with the EHR was recorded using the Morae recording software. The software recorded screen capture data, mouse movements, and keystroke actions. Participants' demographic data including years of clinical experience and EHR use were collected using a questionnaire.

The screen capture videos were examined for each MRI order. The data for nine participants (two participants from site 1A, three participants from site 2A, one participant from site 3B, and three participants from site 4B) could not be analyzed due to data capture failure. To map clinical workflow, two human factor experts conducted a detailed task analysis for the workflow pathway used by each physician. Workflow patterns were evaluated using objective performance metrics, including time to completion and error rates. Task time (time to completion) and task completion (accuracy of order) were recorded.

An emergency medicine physician categorized the participants' MRI orders into the following four categories to describe the accuracy of the order: (1) “success” if the participant completed the correct order with the indication of cord compression; (2) “work around/usability” if the participant completed the correct order with the indication of cord compression but had to use a workaround such as writing out instructions in a field that may be missed by the radiologist; (3) “clinically equivalent” if the participant completed the correct order with an indication of back pain or an indication other than cord compression; and (4) “fail” if the participant failed to order all of the requested studies (cervical, thoracic, and lumbar MRI) or placed a wrong order (e.g., a different test or imaging modality). A consensus-based approach was applied to determine workflow sequence and judgment of accuracy across experts. Statistical analyses were conducted using “rcompanion_1.13.2,” ”vcd_1.4–4,” ”lsr_0.5” in R 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria).[17]

Results

A total of 55 emergency medicine physicians, including attendings and residents, participated in the study. Participant demographics are shown in [Table 1].

Table 1 Participant demographics by site (55 participants)

	Site 1A (N = 14)	Site 2A (N = 15)	Site 3B (N = 14)	Site 4B (N = 12)
Role	Count	Count	Count	Count
Attending	5	15	8	7
Resident	9	0	6	5
Experience in role (years)
1–5	12	5	11	7
6–10	1	9	1	1
11+	1	1	2	4
Experience with EHR (years)
Mean (standard deviation)	2.89 (1.49)	6.8 (5.09)	3.4 (1.05)	3.12 (1.61)

Abbreviation: EHR, electronic health record.

Mapping Clinical Workflow

Analysis of the physician's interaction with the EHR indicated that physicians follow different workflows to complete an MRI order ([Fig. 1]). The task required (1) placing orders for spinal segment MRIs and (2) modifying each order with an indication. This resulted in the following four workflow patterns:

• “Six-step sequential”: first, the user places one order for either cervical, thoracic, or lumbar MRI. Second, the user modifies that single order. Next, the user places an order for a second spinal segment MRI and then modifies that second order. Finally, the user places an order for the third spinal segment MRI and modifies the final order. This workflow includes six steps.

• “Six-step serial”: first, the user places three orders, one for each spinal segment MRI (cervical, thoracic, and lumbar). Next, the user modifies each of the three orders separately. This workflow includes six steps and multiple task switches.

• “Four-step batch”: first, the user places three orders, one for each spinal segment MRI (cervical, thoracic, and lumbar). Next, the user modifies all three orders in a single modification (batch modification). This workflow includes four steps.

• “Two-step batch”: first, the user employs functionality built into the system to place orders for all three spinal segments in a single step (batch order). Next, the user modifies all three orders in a single modification (batch modification). This workflow includes two steps.

Fig. 1 Workflow patterns based on the differences in the way participants select or modify parts of MRI test order. ED, emergency department; MRI, magnetic resonance imaging.

Variability in Clinical Workflow

All sites differed in the workflow options used by physicians to complete the same task ([Fig. 2]). There was a significant difference in the workflow options adopted by physicians at different sites to complete the MRI order (p = 0.002; Fisher's exact test). We observed that only physicians at site 1A used the option to modify all orders together (“four-step batch”). Only physicians at site 3B used the option to order and modify all three MRI tests together (“two-step batch”). Physicians at sites 2A and 4B used workflow option “six-step sequential” or “six-step serial.”

Fig. 2 Frequency of workflow usage across four health care systems.

Accuracy and Workflow

The distribution of clinical correctness (accuracy) varied across workflow options. [Fig. 3] shows the counts of correctness using each workflow option. We found a moderate-to-high interaction between the workflow option used and the correctness of the order (Cramer's V = 0.412; p = 0.002; Fisher's exact test). [Table 2] shows pairwise comparisons of workflow option and clinical correctness of the order, demonstrating statistically significant differences in the clinical correctness achieved by physicians who employed each workflow option. The “two-step batch” approach led to significantly higher clinical correctness of the order compared with all other workflow options.

Fig. 3 Frequency of clinical performance results based on workflow pattern.

Table 2 Posthoc workflow: clinical correctness comparison using Fisher's exact test

Comparison	p-Value	Adjusted p-value
Two-step batch: four-step batch	0.006	0.017
Two-Step Batch: six-step serial	0.004	0.017
Two-Step Batch: six-step sequential	0.009	0.019
Four-step batch: six-step serial	0.812	0.974
Four-step batch: six-step sequential	0.702	0.974
Six-step serial: six-step sequential	1.000	1.000

Note: Significant p-Values are depicted in bold.

Time and Workflow

[Fig. 4] shows the mean and standard deviation (SD) of task time using different workflow options. Workflow option “two-step batch” took the shortest time to complete the order (mean = 29.65 seconds; SD = 13.78). One-way analysis of variance (ANOVA) showed significant differences in mean task time between workflow options (p < 0.001). Posthoc analysis using Tukey's Honest Significant Difference showed significant pairwise differences ([Table 3]).

Fig. 4 Workflow and time.

Table 3 Tukey's Honest Significant Difference test: pairwise differences in task time between workflow options

Workflow	Estimated mean difference in time and confidence interval	p-Value
Four-step batch: two-step batch	43.21 (11.66–74.76)	<0.001
Six-step serial: two-step batch	57.84 (32.15–83.54)	< 0.001
Six-step sequential: two-step batch	27.35 (0.27–54.43)	0.047
Six-step serial: four-step batch	14.64(–15.73 to 45.00)	0.575
Six-step sequential: four-step batch	–15.86 (–47.41 to 15.69)	0.541
Six-step sequential: six-step serial	–30.49 (–56.19 to –4.80)	0.014

Note: Significant p-Values are depicted in bold.

Clicks and Workflow

[Fig. 5] shows the mean and SD of the number of clicks using each workflow options. Workflow option “two-step batch” took the lowest number of click equivalents (clicks, tabs, wheel scroll, enter) to complete the order (mean = 13.50; SD = 18.87), whereas “six-step serial” took the maximum number of clicks equivalents (mean = 34.60; SD = 14.45). One-way ANOVA showed significant differences in mean task time between workflow options (p = 0.002).

Fig. 5 Workflow and clicks.

Discussion

In this study, we found that the time required to complete a standardized ordering task and the clinical accuracy of that order varied based on the EHR workflow used by physicians. Furthermore, affordances built into different EHR systems influenced physicians' choice of workflow. While physicians who used the “two-step batch” workflow spent less time on orders, required fewer clicks, and had improved accuracy, only one of the four sites had designed their EHR to enable this workflow. In contrast, at two sites, the only possible workflows required six steps, leading to increased time on task and clicks, as well as reduced accuracy. Ultimately, these results demonstrate that EHR design can impact the time to complete tasks and their accuracy, which can affect patient safety as well as clinician frustration and burnout.

For each site, only some of the four workflow options were used. These may be determined by changes made during product implementation at a particular site and further local customizations and modifications. The different workflows identified in placing an MRI order suggest that a physician who works at multiple sites with different EHRs or instances of the same EHR would likely have to adapt to different workflows for doing the same task. The introduction of different workflows for the same task may contribute to increased cognitive load, provider frustration, lack of awareness of system capability or system state, and the inability to optimize task sequence or workflow. Successful EHR implementation requires that end users understand each task-specific workflow (e.g., MRI order), all technology components work properly with the corresponding workflow, and each end user knows what options are available through relevant software components. While local customizations are often needed to support specific health system workflows, usability testing is rarely conducted to evaluate workflow implications. The application of human factor engineering and user-centered design are just two approaches that could help health care systems reduce inefficiencies and improve patient safety.

This study has a few limitations. We evaluated EHR interactions of only a limited number of physicians. We may have missed other potential workflow pathways due to our limited sample size. The study was conducted in an office setting rather than a clinical environment, which may impact fidelity of the study with regard to added stress and other compounding factors common in the clinical environment, such as interruptions. The study considered only MRI orders, and the findings may not extrapolate to other ordering tasks. However, many other orders also have variations in workflows similar to MRI, where some options involve more steps and task switches that can negatively impact care. Moreover, we did not include data on potential confounders such as the organizational factors, EHR training differences, or other contextual factors that may have influenced performance metrics. Nonetheless, given the large effect sizes found in this study, we feel that the workflow chosen is likely the primary determinant of time on task and clinical correctness. Finally, isolated usability tests for a single task within a larger workflow cannot speak of how the variations in that single order impact the overall workflow of patient care.

The introduction of EHRs has overall demonstrated reduced errors, improved patient safety, and supported better patient outcomes, but it has also given rise to new, unanticipated safety challenges.[18] [19] [20] Researchers have identified similar issues in the clinical workflow challenges outlined in our study as “goodness-of-fit” issues in the user-friendliness of EHR systems.[9] [21] Several high-profile reports have called for multidisciplinary approaches to reduce health information technology (IT) related and EHR-related harm to patients. This may be accomplished by establishing a risk-based regulatory framework, developing health IT safety collaboratives including vendors, leveraging policy regulations such as HITECH (Health Information Technology for Economic and Clinical Health) Act to incentivize health IT use, and a general focus on usability as a critical component of safe and effective use of health IT.

Conclusion

This study demonstrates significant variability in the clinical workflow that contributes to inaccurate orders as well as more time and clicks required. By optimizing usability and workflow patterns, the system can better support physicians, leading to improved accuracy and reduced task completion time, which, in turn, can reduce clinician frustration and improve patient safety. The variability highlights the critical impact of the local implementation of an EHR on the ultimate usability and safety of each EHR installation. This research demonstrates that further work is needed to optimize EHRs based on workflow and usability.

Clinical Relevance Statement

EHR ordering systems have a wide variation in workflow options for physicians to place orders, which impacts both time and safety. Our research highlights the impact of different workflow options, namely time to task completion and accuracy. Identifying and standardizing workflows within the EHR can improve and optimize patient care and reduce inefficiencies. This study represents a larger goal to design technology to fit provider workflow rather than training individuals to adapt to poorly designed technology.

Conflict of Interest

None declared.

Acknowledgments

This study was supported by a contract from Medstar Health.

Protection of Human and Animal Subjects

This study was approved by the Institutional Review Board.

• References

• 1 Likourezos A, Chalfin DB, Murphy DG, Sommer B, Darcy K, Davidson SJ. Physician and nurse satisfaction with an Electronic Medical Record system. J Emerg Med 2004; 27 (04) 419-424
• 2 Friedberg MW, Chen PG, Van Busum KR. , et al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy. Rand Health Q 2014; 3 (04) 1
• 3 McGinn CA, Grenier S, Duplantie J. , et al. Comparison of user groups' perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med 2011; 9 (01) 46
• 4 Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015; 22 (06) 1179-1182
• 5 Howe JL, Adams KT, Hettinger AZ, Ratwani RM. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018; 319 (12) 1276-1278
• 6 Niazkhani Z, Pirnejad H, Berg M, Aarts J. The impact of computerized provider order entry systems on inpatient clinical workflow: a literature review. J Am Med Inform Assoc 2009; 16 (04) 539-549
• 7 Lee J, Cain C, Young S, Chockley N, Burstin H. The adoption gap: health information technology in small physician practices. Understanding office workflow can help realize the promise of technology. Health Aff (Millwood) 2005; 24 (05) 1364-1366
• 8 American College of Physicians. Survey of clinicians: user satisfaction with electronic health records has decreased since 2010. Available at: https://www.acponline.org/acp-newsroom/survey-of-clinicians-user-satisfaction-with-electronic-health-records-has-decreased-since-2010 . Accessed September 2018
• 9 Middleton B, Bloomrosen M, Dente MA. , et al; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1): e2-e8
• 10 Sinsky C, Colligan L, Li L. , et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
• 11 Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care--an interactive sociotechnical analysis. J Am Med Inform Assoc 2007; 14 (05) 542-549
• 12 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; 19 (Suppl. 03) i68-i74
• 13 Park SY, Lee SY, Chen Y. The effects of EMR deployment on doctors' work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform 2012; 81 (03) 204-217
• 14 Vishwanath A, Singh SR, Winkelstein P. The impact of electronic medical record systems on outpatient workflows: a longitudinal evaluation of its workflow effects. Int J Med Inform 2010; 79 (11) 778-791
• 15 Unertl KM, Weinger MB, Johnson KB, Lorenzi NM. Describing and modeling workflow and information flow in chronic disease care. J Am Med Inform Assoc 2009; 16 (06) 826-836
• 16 Ratwani RM, Savage E, Will A. , et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018; 25 (09) 1197-1201
• 17 R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2018
• 18 Blumenthal D. Stimulating the adoption of health information technology. N Engl J Med 2009; 360 (15) 1477-1479
• 19 Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood) 2009; 28 (02) w282-w293
• 20 Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood) 2011; 30 (03) 464-471
• 21 National Research Council. Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions. Washington, DC: National Academies Press; 2009