Efficiency of Emergency Physicians: Insights from an Observational Study using EHR Log Files

Abstract

Objective With federal mandates and incentives since the turn of this decade, electronic health records (EHR) have been widely adopted and used for clinical care. Over the last several years, we have seen both positive and negative perspectives on its use. Using an analysis of log files of EHR use, we investigated the nature of EHR use and their effect on an emergency department's (ED) throughput and efficiency.

Methods EHR logs of time spent by attending physicians on EHR-based activities over a 6-week period (n = 2,304 patients) were collected. For each patient encounter, physician activities in the EHR were categorized into four activities: documentation, review, orders, and navigation. Four ED-based performance metrics were also captured: door-to-provider time, door-to-doctor time, door-to-disposition time, and length of stay (LOS). Association between the four EHR-based activities and corresponding ED performance metrics were evaluated.

Results We found positive correlations between physician review of patient charts, and door-to-disposition time (r = 0.43, p < 0.05), and with LOS (r = 0.48, p < 0.05). There were no statistically significant associations between any of the other performance metrics and EHR activities.

Conclusion The results highlight that longer time spent on reviewing information on the EHR is potentially associated with decreased ED throughput efficiency. Balancing these competing goals is often a challenge of physicians, and its implications for patient safety is discussed.

Keywords

physician performance - EHR logfiles - meaningful use - emergency department

Background and Significance

Electronic health records (EHR), long seen as a key for improving health care processes, have received increased attention over the last decade.[1] [2] This is because EHRs support several functional aspects of clinical care, including (1) documentation of patient care,[3] (2) clinical decision making and reasoning,[4] [5] [6] (3) computerized provider order entry,[7] (4) information search,[8] and (5) communication between care providers.[9] However, widespread adoption and use of EHRs has been met with significant challenges,[10] [11] partially due to factors such as their usability,[12] increased implementation and maintenance costs,[13] [14] limited integration with clinician workflow, and other unintended consequences.[15]

To stimulate EHR adoption and standardize its use, the Health Information Technology for Economic and Clinical Health (HITECH) act of 2009 offered financial incentives for the implementation and “meaningful use” of EHRs. Toward this end, the expectation was that such Meaningful Use (MU) would, in turn, improve clinical quality measures, thereby improving clinical outcomes for patients. In response to HITECH, health care practices, including hospitals, outpatient centers, and ambulatory care settings, have implemented EHR systems. Although there have been several positive reports on the impact of EHR use on patient outcomes, confirmatory evidence regarding the use of EHRs is still lacking. For example, Park et al's study in an emergency department (ED) found that EHRs increased the cognitive burden of physicians and reduced their direct interaction time with patients.[16] However, other researchers have described the impact of EHRs on improving cost-effectiveness for care,[17] management of privacy,[18] [19] and improvements in chronic care.[20]

More interestingly, what is less known is how the necessity to comply with these requirements has potentially affected how clinicians perform their routine activities. In other words, despite the widespread implementation and adoption of EHRs, very little is known about their direct impact on clinical activities and workflow.[4] [21] There are potentially several reasons for such limited research in this space. First, traditionally, much of the research evaluating clinical workflow has relied on the use of observational approaches.[22] Over the last decade, several studies used these approaches—ranging from field observations, to structured shadowing sessions[22] [23]—to characterize the nature, characteristics, and effects of workflow. Although these studies have provided significant insights regarding clinical workflow activities in a variety of settings, these efforts are time-consuming and require significant effort in planning and data collection.[8] [24]

More recently, alternative mechanisms relying on automated data collection techniques, including the use of radiofrequency identification systems, Bluetooth, or other real-time location sensing devices have been used to track clinical workflow activities.[25] [26] These studies have led to more streamlined, and quicker observational studies. However, sensor techniques alone are often unable to capture the in-depth contextual information often required to characterize clinical workflow.

One of the alternatives, albeit an approximate one, is to use techniques of computational ethnography[27] to collect, analyze, and interpret computer-generated logs of EHR activities of clinicians. These logs provide a trace of actions undertaken by clinicians during their patient care process. For example, Hripcsak et al[28] used EHR data logs to characterize the documentation practices of clinicians. Similar efforts to track clinical documentation and activities provided insights regarding EHR use in general.[20] [29] However, the use of such techniques has been limited at best.

In this paper, our focus is on the use of EHR log files to characterize the effects of EHR use on performance metrics in the ED. Specifically, we utilize EHR usage logs to: (1) identify the nature of activities performed by attending physicians, (2) calculate the time spent on each of these activities, and (3) evaluate the relationship of these activities to performance and workflow effectiveness.

Methods

Study Setting and Participants

This study was conducted in the adult ED at the Mount Sinai Hospital in New York, New York, United States. This ED serves as a transient facility where patients are stabilized and evaluated, then either discharged or admitted to the hospital for further management. The ED provides care to an underserved, inner city, and primarily low-income priority population. Nearly 100,000 patients are seen in the adult ED annually, and ∼27% of these patients are admitted to the hospital.

The adult ED unit comprises 44 monitored beds, gynecology (GYN) rooms with portable ultrasonography, negative pressure rooms, an orthopedic cast care area, an asthma area, a five-bed resuscitation room, and a Hazmat (hazardous material) decontamination area. The ED follows a shift-based schedule with three attending physcician shifts (7 a.m.–3 p.m., 3–11 p.m., 11 p.m.–7 a.m.) where 2 to 4 attending physicians are in the ED at any given time. ED clinicians used Epic ASAP 2014, an ED-specific interface which coordinates ED visits, tracking information about room occupation and displaying bed, and room status on monitors mounted near the nursing station.

Eight (n = 8) ED attending physicians provided written consents to have their patient deidentified EHR activity logs used for this study. Residents, physician assistants, or nurses were not included in this study.

Data Collection

The primary source of data for this study was the EHR use logs from consented attending physicians. These data logs represented the actions of physicians on the EHR interface. For example, when a physician orders a medication or writes a progress note, these actions are stored in the database as an audit trail. All EHRs currently have some version of such functionality and such data are often used for audit reviews.

Using a standard reporting tool, we retrieved physician activities for eight consented attending physicians for a 6-week period between December 2015 and January 2016. For each session, we collected the following information from each attending physician: (1) clinician ID: clinician using the EHR (a sequential number, with no identifying information regarding the clinician); (2) patient ID: a deidentified patient identifier in sequential order; (3) source(s) accessed: this includes a set of 400 actions that were performed in the EHR; (4) start and end time of access: time at which the access to the source started and ended; a pseudo time that is shifted based on a random number-based shift (e.g., a 15-day, 25-minute shift). No patient identifying information, their clinical conditions, or treatment decisions were retrieved or used for analysis.

We only included physician activities of patients who were discharged from the ED and those who had moderate acuity level, 3 to 5 on the 5-point Emergency Severity Index (ESI).[30] This was done to control for variation in patient acuities, which can lead to differences in the time required to care for the patient. For example, it is likely that patients with high acuity conditions are admitted to a hospital unit. Based on our discussion with the ED physicians, patients with lower ESI scores were also more likely to be discharged from the ED.

In addition to this, for each patient encounter, we captured the following ED-based performance measures: door-to-provider time, door-to-doctor time, door-to-disposition time, and the length of stay (LOS) in the ED.

Door-to-provider time represents the time from patients' arrival in the ED until any clinical provider sees that patient (e.g., a triage nurse). Door-to-doctor time is the time from patient arrival to physician encounter, including a resident or physician assistant, who often see patients before the attending (NQF ED #005–08). Door-to-disposition is the time from patient arrival to when a disposition or discharge decision is made (NQF ED #002–08). LOS is the total time a patient spends in the ED from their arrival in the unit (NQF ED #007–08).

Coding Clinician Activities

Based on the EHR usage logs, we identified a total of 138 activities. These included short activities such as clicking a button to exit a patient record as well as longer-term activities such as writing a note or ordering a test or medication. See [Table 1] for a list of relevant examples.

Table 1 Subcategorization of EHR activities and their related activities

EHR activity	Definition	Encompassing activities
Documentation	Clinical activities such as writing clinical notes during or after a patient encounter	Notes section, allergies, result entry
Review	Viewing charts, previous encounters, or vitals via the flow sheet	Results review, history, snapshot visit
Order	Submitting and accessing of all orders	Medication order, laboratory order, order reconciliation
Navigation	Usage of search functions as well the transition between various screens within the EHR system	Chart search, patient list, exit record

Abbreviation: EHR, electronic health record.

Given the granular nature of these activities, we categorized 138 activities into 4 generic higher-level categories: documentation, review, orders, and onscreen navigation. As part of a larger study, this categorization was derived from ethnographic observations and shadowing of clinician activities, semistructured interviews with ED clinicians, and discussions with an ED physician collaborator.[31]

Documentation encompassed clinical activities such as writing clinical notes during or after a patient encounter. Attestation of resident or physician assistants' notes by attending physicians were also included in this category. Review included viewing charts, results, previous encounters, or vitals via the flow sheet. The order category was comprised of all activities related to an order including creating, submitting, changing, or canceling orders (e.g., medications, laboratories, consults). Finally, navigation included the usage of search functions as well the transition between various screens within the EHR system (e.g., exiting orders).

This categorization and coding was conducted in an iterative fashion. First, we preliminarily classified all activities in high-level groups (e.g., documentation, orders). Then, with the help of two ED physicians, these activities were categorized into the four abovementioned groups.

Data Analysis

Data analysis was conducted in several stages. First, EHR-based activities were categorized according to the four groups: documentation, review, orders, and navigation. Second, for each patient, each of the ED performance metrics (door-to-provider, door-to-doctor, door-to-disposition, and LOS) were matched with the corresponding attending physician's EHR usage activities. With this matching, we were able to organize the time spent on each of the four EHR activities for a patient as compared with each of the performance metrics.

At the end of this processing, for each patient, each of their performance measurements were categorized based on the corresponding time spent on four activities. [Fig. 1] shows the workflow that was used to organize the ED performance metrics and the EHR-based activities for each of the patients. For a small sample of patients (n = 20), we were unable to perform this matching because of missing details regarding one or more of the ED performance metrics or EHR-based activity data. These were removed for analysis.

Fig. 1 Patient trajectory from arrival to the emergency department (ED) to discharge. The four ED performance metrics: door-to-provider, door-to-doctor, door-to-disposition, and length of stay are shown with respect to a patient's care trajectory through the ED. On the lower half, the time spent on each of the four electronic health record (EHR)-based activities during each of the performance metric measurement phases is shown: documentation (D), review (R), orders (O), and navigation (N). The time spent measures are not drawn to scale.

Pearson's correlation coefficients were computed between each of the performance metrics and each of the four EHR activities. All statistical analysis was conducted using the R statistical software. An α of 0.05 was used to test for statistical significance, unless otherwise specified.

Results

We had a total of 2,304 patients in the data set over a 6-week period. On average, the physicians spent most time on documentation per patient (M = 347 s [SD = 38.4 s]), followed by review (291 s [78.1 s]), orders (143 s[62.1 s]), and navigation (15 s [13.4 s]).

As previously described, we mapped the time spent on clinical activities with the four considered ED-based performance measures: door-to-provider, door-to-doctor, door-to-disposition, and LOS. For each metric, we computed the time spent on each of the four considered EHR-based activities, and then computed the Pearson's correlation coefficient between the time spent on the EHR-based activities and the corresponding ED-based performance metric.

Based on the analysis, we found that there were positive correlations between physician time taken to review patient charts and door-to-disposition time (r = 0.43, p < 0.05), and with LOS (r = 0.48, p < 0.05). There was no statistically significant correlation between any of the other performance metrics and the EHR activities.

Discussion

Based on an analysis of EHR logs files of physician activities in an urban ED, we found that there was a possible association between a physician's EHR-based activities and the overall efficiency of the ED. In other words, we found potential associations between the time it took for physicians to review patient information and the time taken to reach a patient disposition, and correspondingly the patient's LOS in the ED. Although this does not mean that there is a causal effect, such a potential association warrants further investigation.

Two interesting insights can be drawn from these results. First, the associations highlight the role of an attending physician in reviewing patient information in the EHR, and how that may potentially affect the time taken for patient disposition decisions and discharge. In some ways, this can be argued as a default clinical workflow process in an academic ED. In other words, academic medical centers have a multilayered process of iterative review with residents performing the frontline of clinical activity, supervised and guided by attending physicians. This helps in possibly improving the safety of patient care. The downside, as highlighted from our results, is the potential impairment in performance (in terms of longer time involved in the care process).

Second, it also highlights the nature of care in team-based distributed environments such as the ED. With multiple clinicians, residents, nurses, pharmacists, and attending physicians involved in the care of the patient, patient data are updated in a nonlinear fashion. As a result, final reviews and decisions are potentially delayed until all such information is available. In other words, often times, attending physicians wait until all pertinent patient information (e.g., test results or imaging results) are available to make patient disposition decisions. However, it must also be acknowledged that other factors, besides what we discussed, may also influence such decisions.

In summary, it is important to consider the balance between efficiency and safety. Clinical settings have been transformed into high-intensity environments where work practices are rooted in the dual goals of performance efficiency and safety. However, achieving these competing aims is one of the greatest challenges of modern clinical environments. The question then becomes how the clinician balances these dual goals of performance and safety, without compromising either of these goals.

The results also have implications for characterizing the effect of EHR use on clinical workflow efficiency and clinical performance. Recent research has highlighted the significant amount of time that physicians spent on the EHR in comparison to patient care.[32] [33] Our results highlight how the percentage of time spent on specific EHR-based activities potentially impacted the overall throughput in an ED. However, it is also interesting to note that we did not find potential associations between the time spent on EHR documentation and door-to-disposition or LOS. As reported in previous research,[32] [33] the longer time required to fulfill documentation requirements on the EHR can increase physician effort and cognitive load. Although previous research, conducted in outpatient settings, did not measure the effect of longer times spent on EHR on efficiency metrics, this may be a potential area for further exploration and research.

Finally, there are potential challenges in the use of EHR-based logs for conducting research on clinical and process efficiency. As highlighted from our study, we were able to utilize a large amount of data and use it to generate insights regarding EHR use. At the same time, there were significant challenges in the access, transformation, and use of this data. We needed significant support from the hospital and ED management. The biggest challenge was anonymizing patient information and matching it to specific performance metrics.

Limitations

We would like to acknowledge some limitations of our study. Our analysis considered only the EHR activities of the attending physicians. Other clinicians also perform similar tasks in the EHR for each patient increasing the total time spent per patient. We chose to focus on the attending physician as they are ultimately responsible for all decisions that are made regarding a patient. However, it should also be acknowledged that an attending physician's workflow is also determined by other clinicians' activities. Additionally, the choice of attending physicians was made such that we had uniformity in terms of our participants. This study was based on a small sample of physicians and a very large sample of patients (n = 2,304) at a single academic center. The ED-based performance measures that we considered were incremental and colinear. In other words, the LOS time is the aggregate of the other three ED-based performance metrics. However, given that we computed EHR-based activities separately, for each segment of the ED-based performance metrics, the potential associations are still valid. We did not capture the patient demographics or their clinical conditions for our analysis. Accounting for the differences in clinical conditions in the analysis may, more than likely, weaken the associate effects that were observed. Additionally, we only considered patients who presented with lower ESI, and those who were discharged from the ED. This may have excluded sicker patients, who may require different patterns of EHR usage. We also used automatically derived metrics for the ED performance measures. These measures were based on EHR-based events; for example, when a vital sign was recorded, or when a physician created a note. Finally, we used a range of patients in the analysis (ESI score, 3–5), and these patients may have had varied clinical requirements and complexity. As a result, the amount of time spent on these patients varied considerably.

Conclusion and Future Directions

In this paper, we report on a study investigating the association between EHR-based activities and ED-based performance measures. The results highlight the potential associations between attending physicians' EHR activities and patient dispositions. As previously mentioned, these associations are not meant to show causal effects, but provide opportunities for further research. Further research is needed to establish how clinicians use the EHR to manage patient care activities. As part of a larger research project on how EHRs affect clinical workflow in the ED, we are conducting a multimethod study supported by ethnographic observations, shadowing of clinicians, and follow-up interviews. These studies can provide insights into how clinicians integrate EHR activities into their workflow, the challenges they face, and opportunities for improving the interface and workflow aspects of ED-based patient care.

Multiple Choice Question

EHR logs provide a trace of the clinician's activities during patient care. In addition to providing an overview of how a clinician performs their tasks, it also gives a snapshot of the strategies and behaviors. However, it has some disadvantages. Which one of the following is a significant disadvantage of using EHR logs for research tracing the EHR-based activities of a clinician?

• Does not account for the significant cognitive actions and social interactions that contribute to care

• Data logs do not provide PHI

• Data logs are unreliable

• Data logs cannot be obtained for all clinician interactions on the EHR

Correct Answer: The correct answer is d, EHR logs, while providing significant insight into the clinician activities, provide limited insight on the context of care and clinician's cognitive processes.

Conflict of Interest

None.

Protection of Human and Animal Subjects

No subjects were recruited to participate in this study. The study was based on retrospective analysis of EHR logs. This study was approved by the institutional review board at the study site.

• References

• 1 Blumenthal D. Stimulating the adoption of health information technology. N Engl J Med 2009; 360 (15) 1477-1479
• 2 Dick RS, Steen EB. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington, DC: Institute of Medicine; 1991
• 3 Häyrinen K, Saranto K, Nykänen P. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform 2008; 77 (05) 291-304
• 4 Patel VL, Kannampallil TG. Human factors and health information technology: current challenges and future directions. Yearb Med Inform 2014; 9 (01) 58-66
• 5 Patel VL, Kannampallil TG, Shortliffe EH. Role of cognition in generating and mitigating clinical errors. BMJ Qual Saf 2015; 24 (07) 468-474
• 6 Patel VL, Kaufman DR, Kannampallil TG. Diagnostic reasoning and decision making in the context of health information technology. Rev Hum Factors Ergon 2013; 8 (01) 149-190
• 7 Geisler BP, Schuur JD, Pallin DJ. Estimates of electronic medical records in U.S. emergency departments. PLoS One 2010; 5 (02) e9274
• 8 Kannampallil TG, Jones LK, Patel VL, Buchman TG, Franklin A. Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings. J Am Med Inform Assoc 2014; 21 (e2): e249-e256
• 9 Shapiro JS, Mostashari F, Hripcsak G, Soulakis N, Kuperman G. Using health information exchange to improve public health. Am J Public Health 2011; 101 (04) 616-623
• 10 Ford EW, Menachemi N, Phillips MT. Predicting the adoption of electronic health records by physicians: when will health care be paperless?. J Am Med Inform Assoc 2006; 13 (01) 106-112
• 11 Hersh W. Health care information technology: progress and barriers. JAMA 2004; 292 (18) 2273-2274
• 12 Middleton B, Bloomrosen M, Dente MA. , et al; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1): e2-e8
• 13 Simon SR, Kaushal R, Cleary PD. , et al. Correlates of electronic health record adoption in office practices: a statewide survey. J Am Med Inform Assoc 2007; 14 (01) 110-117
• 14 Wright A, Henkin S, Feblowitz J, McCoy AB, Bates DW, Sittig DF. Early results of the meaningful use program for electronic health records. N Engl J Med 2013; 368 (08) 779-780
• 15 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; 11 (02) 104-112
• 16 Park SY, Lee SY, Chen Y. The effects of EMR deployment on doctors' work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform 2012; 81 (03) 204-217
• 17 Ben-Assuli O, Ziv A, Sagi D, Ironi A, Leshno M. Cost-effectiveness evaluation of EHR: simulation of an abdominal aortic aneurysm in the emergency department. J Med Syst 2016; 40 (06) 141
• 18 Ben-Assuli O. Electronic health records, adoption, quality of care, legal and privacy issues and their implementation in emergency departments. Health Policy 2015; 119 (03) 287-297
• 19 Caine K, Hanania R. Patients want granular privacy control over health information in electronic medical records. J Am Med Inform Assoc 2013; 20 (01) 7-15
• 20 Calvitti A, Hochheiser H, Ashfaq S. , et al. Physician activity during outpatient visits and subjective workload. J Biomed Inform 2017; 69: 135-149
• 21 Patel VL, Kannampallil TG. Cognitive informatics in biomedicine and healthcare. J Biomed Inform 2015; 53: 3-14
• 22 Malhotra S, Jordan D, Shortliffe E, Patel VL. Workflow modeling in critical care: piecing together your own puzzle. J Biomed Inform 2007; 40 (02) 81-92
• 23 Abraham J, Kannampallil TG, Reddy M. , Eds. Peripheral Activities during EMR Use in Emergency Care: A Case Study. American Medical Informatics Association (AMIA) Annual Symposium; 2009; San Francisco, CA
• 24 Kannampallil TG, Abraham J, Patel VL. Methodological framework for evaluating clinical processes: a cognitive informatics perspective. J Biomed Inform 2016; 64: 342-351
• 25 Kannampallil T, Li Z, Zhang M. , et al. Making sense: sensor-based investigation of clinician activities in complex critical care environments. J Biomed Inform 2011; 44 (03) 441-454
• 26 Vankipuram M, Kahol K, Cohen T, Patel VL. Toward automated workflow analysis and visualization in clinical environments. J Biomed Inform 2011; 44 (03) 432-440
• 27 Zheng K, Hanauer DA, Weibel N, Agha Z. Computational Ethnography: Automated and Unobtrusive Means for Collecting Data In Situ for Human–Computer Interaction Evaluation Studies. Cognitive Informatics for Biomedicine: Human Computer Interaction. London, United Kingdom: Springer; 2015: 111-140
• 28 Hripcsak G, Vawdrey DK, Fred MR, Bostwick SB. Use of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc 2011; 18 (02) 112-117
• 29 Zheng K, Padman R, Johnson MP, Diamond HS. An interface-driven analysis of user interactions with an electronic health records system. J Am Med Inform Assoc 2009; 16 (02) 228-237
• 30 McHugh M, Tanabe P, McClelland M, Khare RK. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med 2012; 19 (01) 106-109
• 31 Denton C, Shapiro JS, Nimo G, Kannampallil TG, Patel VL. , Eds. Perceived Effects of Meaningful Use of Electronic Health Records on Clinical Workflow. AMIA Annual Symposium; 2016; Chicago, IL
• 32 McDonald CJ, Callaghan FM, Weissman A, Goodwin RM, Mundkur M, Kuhn T. Use of internist's free time by ambulatory care Electronic Medical Record systems. JAMA Intern Med 2014; 174 (11) 1860-1863
• 33 Sinsky C, Colligan L, Li L. , et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760