The Impact of a Location-Sensing Electronic Health Record on Clinician Efficiency and Accuracy: A Pilot Simulation Study

Abstract

Background Through the Health Information Technology for Economic and Clinical Health Act of 2009, the federal government invested $26 billion in electronic health records (EHRs) to improve physician performance and patient safety; however, these systems have not met expectations. One of the cited issues with EHRs is the human–computer interaction, as exhibited by the excessive number of interactions with the interface, which reduces clinician efficiency. In contrast, real-time location systems (RTLS)—technologies that can track the location of people and objects—have been shown to increase clinician efficiency. RTLS can improve patient flow in part through the optimization of patient verification activities. However, the data collected by RTLS have not been effectively applied to optimize interaction with EHR systems.

Objectives We conducted a pilot study with the intention of improving the human–computer interaction of EHR systems by incorporating a RTLS. The aim of this study is to determine the impact of RTLS on process metrics (i.e., provider time, number of rooms searched to find a patient, and the number of interactions with the computer interface), and the outcome metric of patient identification accuracy

Methods A pilot study was conducted in a simulated emergency department using a locally developed camera-based RTLS-equipped EHR that detected the proximity of subjects to simulated patients and displayed patient information when subjects entered the exam rooms. Ten volunteers participated in 10 patient encounters with the RTLS activated (RTLS-A) and then deactivated (RTLS-D). Each volunteer was monitored and actions recorded by trained observers. We sought a 50% improvement in time to locate patients, number of rooms searched to locate patients, and the number of mouse clicks necessary to perform those tasks.

Results The time required to locate patients (RTLS-A = 11.9 ± 2.0 seconds vs. RTLS-D = 36.0 ± 5.7 seconds, p < 0.001), rooms searched to find patient (RTLS-A = 1.0 ± 1.06 vs. RTLS-D = 3.8 ± 0.5, p < 0.001), and number of clicks to access patient data (RTLS-A = 1.0 ± 0.06 vs. RTLS-D = 4.1 ± 0.13, p < 0.001) were significantly reduced with RTLS-A relative to RTLS-D. There was no significant difference between RTLS-A and RTLS-D for patient identification accuracy.

Conclusion This pilot demonstrated in simulation that an EHR equipped with real-time location services improved performance in locating patients and reduced error compared with an EHR without RTLS. Furthermore, RTLS decreased the number of mouse clicks required to access information. This study suggests EHRs equipped with real-time location services that automates patient location and other repetitive tasks may improve physician efficiency, and ultimately, patient safety.

Keywords

electronic health records and systems - clinical information systems - emergency medicine - clinical care - interfaces and usability - cognition - socio-technical aspects of information technology: error management and prevention

Background and Significance

The Health Information Technology for Economic and Clinical Health Act of 2009 distributed nearly $26 billion dollars in a nationwide effort to deploy electronic health record (EHR) systems and improve patient safety and improve health care efficiency. Much has been written regarding the failure of these systems to deliver on those promises.[1] [2] [3] EHRs are complex, expensive software products which require substantial financial investment to implement.[4] Yet, the designers of these systems largely neglected to properly design the human–computer interface. This omission, it turns out, is critical. Multiple authors have written about how interface issues are driving an increase in medical errors and decreasing physician and nursing efficiency while imperiling patient care.[5] [6] [7]

The causes of this failure are legion; however, the interface between the emergency physician (human) and EHR is a frequently cited factor.[2] [3] [4] While EHRs have been represented to place patient information, such as history, laboratory findings, and medication lists, at the fingertips of clinicians, physicians continue to struggle to efficiently and safely use these systems.

All EHR systems require physicians to correctly select a patient to either review clinical data or input orders or other information. Unfortunately, for a variety of reasons, incorrect patient selection is a common cause of error.[5] [6] “Wrong-patient-wrong-order” is a common type of error in electronic medical records (EMRs) which can easily result in a wrong-patient selection error in the EMR.[8]

Additionally, there is a growing body of literature urging consideration of human–machine interface issues in EMR systems, including efforts to reduce the number of interactions (mouse clicks are a commonly cited surrogate) required for a provider to successfully interact with an EMR.[9]

Over the past decade, real-time location systems (RTLS) have been used in hospitals to track equipment, patients, and staff members. For tracking technology, RTLSs commonly use radiofrequency identification (RFID), Wi-Fi, infrared (IR), Bluetooth, or a combination.[10] Many of these technologies have trade-offs and limitations, such as RFID, which does not require line of sight but has poor location accuracy[11] and may not be able to track specific patients in crowds.[12] Note that there are many possible technologies for real-time location tracking, each of which may be more effective depending on the application. Thus, we define RTLS to be any technology that facilitates real-time indoor tracking of objects and people.

Studies have shown RTLS to be effective in improving workflow and efficiency in hospitals for numerous tasks. Ohashi et al demonstrated in a pilot project that an RTLS could reduce the time nurses spent performing verification tasks.[13] [14] [15] [16] Stübig et al showed that a Wi-Fi-based RTLS decreased treatment time for outpatients by 23%.[17] Stahl et al evaluated an RFID system in a primary care facility and found mixed results. Although the RTLS system reduced overall flow time, waiting room time increased.[18] Stahl et al later introduced real-time feedback based on RTLS data and investigated how it affected physician behavior. They found that for some physicians the amount of face time with patients significantly increased.[19] This demonstrates that RTLS can change physician behavior, if physicians are presented with real-time feedback. However, few scientific studies have evaluated an integrated RTLS-EHR system's effect on EHR usability, physician efficiency, and error in locating patients

Goals of This Investigation

The purpose of this study is to pilot test an EHR with a location-sensing feature and determine its effects on physician efficiency—the time needed to locate a patient, the number of rooms searched, and the number of number of clicks (human–computer interactions)—and physician accuracy—patient identification accuracy—with the EHR interface. Specifically, we compare activated real-time location services (RTLS-A) with disabled real-time location services (RTLS-D) while using a simulated EHR. The results of this simulation pilot study can be used to inform the development of future EHR systems, which could reduce clinician workload and improve efficiency and safety.

Methods

RTLS-A System Description

To provide real-time location services, our system consists of a series of wirelessly networked smartphones, at least one in each clinical room, which track unique quick response (QR) codes affixed to patients (i.e., as a patient identifier) and physicians (i.e., as a physician identifier) ([Fig. 1]). Each phone runs a custom application, based on Unity3d and Vuforia—a library of algorithms for real-time position and orientation tracking of QR codes. The phones constantly run this application, which searches for and tracks visible QR codes in their camera image of the room at approximately 30 Hz. When a QR code is found, the phone updates the application server with the location (i.e., in a one-time setup, phones are preconfigured to know the room number). The server updates the physician's hand-held tablet at 30 Hz and with an approximate network latency of 30 ms, which gives the physician real-time updates of patient location.

Fig. 1 Location-sensing electronic health record (EHR) system. The smartphones optically track the quick response (QR) codes and send the position data to the tablet. Based on this position data, the tablet automatically shows the EHR data of the patient who is in the same room as the physician holding the tablet.

Moreover, when the system detects that a physician enters a room with a patient, the physician's tablet automatically displays the correct patient's EHR, based on the physician's proximity to the patient. Thus, the physician has no need to scan a QR code because the system (i.e., utilizing the network of smartphone cameras) tracks the real-time location of the QR codes attached to patients, rooms, physicians, and anything else that needs to be tracked, such as medical equipment. The primary reasons we chose to use this approach to an RTLS system is that (1) it could be quickly implemented and (2) it was relatively low cost. The aforementioned human–computer interface design could be applied to many other types of RTLS systems, such as IR, RFID, or Wi-Fi.

The main limitation of the system is that it is affected by lighting conditions and requires a line of sight between the cameras and the QR codes. Thus, with many people in a room, there is an increased chance of QR codes being occluded. To minimize occlusion, the system can integrate additional smartphones at different angles and use fluorescent lighting, as typically found in hospitals. At the time of the study, we used only one smartphone per room and one patient per room to control the conditions. Thus, we did not track patients or physicians outside of the rooms. In the future, we plan to investigate multiple patients per room and track markers outside of the rooms. An additional limitation is the resolution of the smartphone camera. Low resolution cameras will require a larger QR code. In this case, the codes used in the study were 8.5″ × 8.5″, which were tracked within approximately 10 to 15 feet in fluorescent lighting. The patient codes were affixed to a static position in each room where the patient was located. The physician codes were attached to the tablet that the participant's carried. In a real hospital setting, the code would be attached to the physician. However, due to the line of sight limitation of the system when using a single camera per room, we attached copies of the code to 5 sides of a cardboard cube on the back of the participant's tablet to minimize occlusion.

All the hardware used in the study was off-the-shelf consumer hardware, which made the base cost of the system relatively low compared with other RTLS systems. Each phone was $150, the router was $90, and the tablet was $250, which makes the cost of our prototype study system with 10 phones $1,840. However, note that if this system was a commercial product, the cost would be much higher due to software development costs, business overhead costs, and profit margin, for example.

Study Design and Setting

We initially created a purpose-designed, notional EHR for this study. The proprietary nature of existing EHRs (such as Cerner or Epic) made modifying an existing system impractical. While our system was never intended to be used clinically, it did replicate a patient room list, a demographics page, and displayed fictional clinical data for each simulated patient, all of which was displayed on the physician's tablet ([Fig. 2]). The simulated EHR was written in Unity and ran on Android tablets and phones. The software was designed to run with the location-sensing feature either enabled or disabled for the purposes of or study

Fig. 2 Simulated electronic health record (EHR). Left, Room list with patient names. Right, Interface to patient specific EHR.

RTLS-A EHR Interface: Unique QR codes were placed in 12′ × 10′ sized rooms to simulate the patients and a QR code was attached to the subjects identifying them as the “physician.” When the location-sensing was enabled, the EHR system detected the location of the “patient” QR codes in relationship to both their physical location and their proximity to the physician. The system created the room list based on the physical room location of the simulated patients ([Fig. 2], left). As the subject entered the room, the system would detect the proximity of the “physician” to the “patient” and display the clinical data for that patient ([Fig. 2], right). When the physician left the room, the location-sensing enabled system automatically returned to the room list.

RTLS-D EHR Interface: When sensing was disabled, the system displayed a static list of patients in rooms. That is, the subject was required to manually tap on the patient name to display the patient's EHR data. The disabled system required the subject to manually tap to change the display to the room list.

A simulated emergency department (ED) was set up in our administrative offices. In both conditions, there was a patient name printed on a piece of paper within each room. This enabled participants to verify the identities of the patients. That is, if the patient names and room did not match the information displayed on the tablet, then the subject knew that the room was incorrect, and they would have to continue the search.

Prior to the commencement of the trial, the investigators input a room list with patient assignments into the location-sensing software for both RTLS-D and RTLS-A. However, to introduce error, the investigators chose to place 4/10 patients in the wrong room during each arm of the trial. During the RTLS-A trial, the software automatically corrected this error, while during the RTLS-D trial, the error persisted.

This study was reviewed and approved by the University of Texas Health at San Antonio Institutional Review Board.

Participants

Ten emergency medicine residents and medical students were recruited for participation in our study during grand rounds and an emergency medicine student interest group meeting. Volunteers were asked to participate in a series of 10 clinical encounters while testing a new EHR system. Volunteers were excluded from participation if they were unable to walk without restriction; unable to read, write, or use a computer system, including a hand-held tablet; and lacked basic medical knowledge or the ability to understand basic medical terms. Simulated patients made of paper were used in clinical encounters with the preselected volunteers. No actual patient data were included in this study.

Interventions

Each volunteer performed three clinically relevant tasks in two trials: the first with the RTLS enabled and then with the RTLS disabled. Between the two trials, the rooms were rearranged in accordance with a room assignment roster for each arm. This prevented the subjects from using memory to recall the location of patients during the second trial. During each trial, subjects performed three major tasks: (1) locate a patient within the department and verify their identity; (2) retrieve requested data for a specific patient; and (3) deliver that information verbally to the patient. In the first trial, the subject first used the RTLS-A EHR and during the second trial used the RTLS-D. A trained investigator verified proper completion of all tasks and recorded the results on a standardized data collection instrument. The number of tasks for each arm and the types of tasks were identical between the two arms. A total of 10 ten subjects participated in the study.

Measurements

Time to Locate Patients: Using a smartphone stopwatch app, we recorded the length of time each subject took to locate the patient as a measure of patient location efficiency.

Number of Rooms Searched: We manually counted the number of times the subject entered a room searching for the patient (number of attempts) as a measure of patient location error.

Number of Mouse Clicks: The EHR software recorded the number of “mouse clicks” the subject entered on their tablet for each trial. This was our measure of EHR user interface usability.

Number of Patients Incorrectly Identified: The number of patients that were incorrectly identified after entering a room. This was an outcome metric of patient identification accuracy. Data sheets were completed by the observers in pen/ink and stored.

Outcomes

Participants were observed throughout each patient encounter and data were recorded for the following variables: time physically locating a patient, number of rooms searched to find a patient, number of patients incorrectly identified, and the number of mouse clicks input into the EHR. Secondary variables such as comments and observations from both subjects and observers were recorded on the data sheets.

Analysis

Prior to conducting the study, we performed a power analysis using the PASS Version 11 software. Assuming a coefficient of variation of 50%, the results of the analysis indicated that it would achieve a 90% power to test with significance level α = 0.05 with N = 5 participants. As 10 participants were readily available, we chose to conduct the study with 10 participants.

Continuously distributed outcomes were summarized with the mean, standard deviation, median, minimum, and maximum. The significance of variation in the mean with device (RTLS-enabled, RTLS-disabled) was assessed with a paired t-test. All statistical testing was two-sided with a significance level of 5%. R was used throughout.

Results

Main Result

Average time to physically locate patients (RTLS-A = 11.9 ± 2.0 seconds vs. RTLS-D = 36.0 ± 5.7 seconds, p < 0.001), average number of rooms searched (RTLS-A = 1.0 ± 1.06 vs. RTLS-D = 3.8 ± 0.5, p < 0.001), and average number of clicks (RTLS-A = 1.0 ± 0.06 vs. RTLS-D = 4.1 ± 0.13, p < 0.001) were significantly reduced with RTLS-A relative to RTLS-D ([Tables 1] and [2]). Time to locate patients is summarized by condition (RTLS-A, RTLS-D) in [Fig. 3]. There were no significant differences between RTLS-A and RTLS-D for patient identification error. All participants made 0 errors in patient identification, in part because the patient name was clearly displayed inside each room.

Fig. 3 In the RTLS-enabled condition, participants took significantly less time to locate patients than in the RTLS-disabled condition.

Table 1 Average time in seconds to locate patients

	RTLS-enabled (N = 10)	RTLS-disabled (N = 10)	Difference (N = 10)	p-Value
				< 0.001
N	10	10	10
Mean ± SD	11.9 ± 2.0	36.0 ± 5.7	–24.1 ± 4.2
Median [Q1, Q3]	11.6 [10.8, 13.2]	37.5 [31.7, 40.8]	–25.4 [–27.0, –21.0]
Min, Max	8.9, 15.7	26.1, 42.7	–29.4, –16.2

Abbreviations: RTLS, real-time location system; SD, standard deviation.

Table 2 Number of attempts to locate patient and click counts

	RTLS-enabled (N = 10)	RTLS-disabled (N = 10)	Total (N = 20)	p-Value
Number of rooms searched				< 0.001
N	10	10	20
Mean ± SD	1.02 ± 0.06	3.82 ± 0.47	2.42 ± 1.47
Median [Q1, Q3]	1 [1, 1]	3.9 [3.65, 4]	2.1 [1, 3.85]
Min, Max	1, 1.2	3, 4.6	1, 4.6
Number of mouse clicks				< 0.001
N	10	10	20
Mean ± SD	1.02 ± 0.06	4.06 ± 0.13	2.54 ± 1.56
Median [Q1, Q3]	1 [1, 1]	4 [4, 4]	2.6 [1, 4]
Min, Max	1, 1.2	4, 4.4	1, 4.4

Abbreviations: RTLS, real-time location system; SD, standard deviation.

Discussion

Results: In our simulated environment, we demonstrate that a location-sensing EHR allows providers to find patients more quickly and with fewer “mouse clicks” than a nonlocation sensing system. Improved provider efficiency coupled with a decreased need to interact with EHR suggest that location-sensing has the potential to decrease overall cognitive burden associated with the use of EHRs in the ED setting.

It was anecdotally observed that when using the RTLS-disabled system, our subjects began to disregard the patient's physical location information on the EHR interface and instead began to rely on a manual search method. Finding workarounds is a well-described adaptation that many clinicians use to overcome the inefficiencies of EHRs.[7] [20] However, this adaptive behavior also can defeat safety systems, which while well-intentioned, can be poorly designed. Medication interaction warnings are an example of a safety system that is often defeated by clinicians who simply succumb to “warning fatigue” and click through multiple warnings presented by a poorly designed EHR, potentially ignoring valid safety issues.[21] [22] [23]

Many experts suggest that human–computer interface issues are impairing the fulfillment of the promise of increased efficiency and improved patient safety, often touted as the driver of widespread deployment of EHRs.[1] [2] [5] [24] Yet, there is a paucity of information regarding the limitations and issues of EHRs. Many contracts between vendors and health care providers such as hospitals, preclude the publication of “negative” reviews or reports of adverse outcomes due to EHRs in the published literature and EHRs are not regulated as medical devices in the United States.[7] [25]

In one of the few studies examining the impact of an EHR design on provider efficiency, Bishop et al used a modifiable EHR system that ED personnel were able to customize to fit their clinical requirements, on demand and without the need for programming. They found that a system that was responsive to the needs of clinicians resulted in fewer mouse clicks (and lower cognitive burden) and improved ED performance metrics.[26]

EDs are consistently urged to be more efficient in evaluating, diagnosing, and dispositioning patients. However, EMRs can harm efficiency and potentially negatively affect patient safety. Our study results suggest that location-sensing systems could enhance providers' efficiency in locating patients while simultaneously decreasing the cognitive burden associated with using EHRs. If health care personnel used location-sensing EHRs at the bedside to input orders, the incidence of “wrong-patient, wrong-order” errors could be dramatically reduced. For example, Han et al investigated the reason behind the increase in pediatric mortality rate from 2.8 to 6.57% after the implementation of computerized physician order entry. They found that a decrease in face-to-face communication and increased time of entering orders away from the bedside may have contributed to this increase in mortality.[27] Furthermore, an EHR that can accurately track the location of the patient could provide a photo onscreen of the patient as health care team enters the room. Additionally, location-enabled features could help decrease the number of human–computer interactions required, decreasing cognitive burden as well as the number of workarounds clinicians may attempt to employ.

Limitations

Our pilot study was conducted in a simulated ED, on simulated patients using a purpose-built, EHR simulation. Therefore, many of the other factors associated with the cognitive burden of clinicians when using EHRs were absent or could not be sufficiently incorporated in our model. While the difference in performance between the location-enabled and nonenabled systems were statistically significant, the actual impact on performance in a real clinical setting is unclear. Further study of location-enabled systems, in a nonsimulated clinical setting, is warranted.

Another limitation is the study design because it was not counterbalanced. That is, all participants first experienced RTLS-A and then RTLS-D, which may have biased the results somewhat. For example, it is possible that participants were more familiar with the room locations when they were using RTLS-D. This may have decreased the overall effect size of our comparison. In the future, we plan to counterbalance the arms when we conduct a larger, less exploratory study.

Conclusion

In our pilot simulation study, we demonstrated that real-time location services reduced time to locate a patient by two-thirds, reduced time in locating patients in physical rooms by 73%, and reduced the number of user–system interactions (mouse clicks) by 75%. These data suggest that a location-aware EHR could reduce the cognitive burden of EHR systems on the end-user by automating key components of the interface, such as selecting patients and displaying the patient-specific data when approaching the patient bedside. Furthermore, by automating processes that are known to be prone to user error, such as patient selection, these systems may improve physician efficiency (e.g., reduce number of rooms searched) and enhance patient safety.

Multiple Choice Questions

1. Which of the following is a leading cause for increased error rates and decreased efficiency when using electronic health records (EHRs)?

   • Poor computing performance.

   • Ill-designed human–computer interfaces.

   • Poor availability of computer workstations.

   • Underfunding by regulatory agencies

   Correct Answer: The correct answer is option b, Ill-designed human–computer interfaces.”Several authors have evaluated the impact of the introduction of EHRs into clinical practice and the results leave much to be desired. Kellermann and Jones wrote that in addition to other factors, “Health IT systems should facilitate the work of clinicians, not hinder it.”

2. Although research in the area of optimizing EHRs for clinical use is scant, which of the following interventions demonstrated improved emergency department performance metrics?

   • Ability for users to alter the interface without advanced programming.

   • Deployment of medical informaticists to customize user interfaces.

   • Purchase of emergency medicine-specific electronic health record systems.

   • Availability of EHR work stations in every patient room.

   Correct Answer: The correct answer is option a, ability for users to alter the interface without advanced programming.” Bishop et al designed a custom EHR that incorporated intense user input in its design with a focus on optimizing the EHR for clinical use. Subsequently, the design of the system allowed the users to iterate their changes easily and perform rapid cycle testing on changes to the interface. This iterative effort resulted in an EHR that was at least 41% more efficient than a legacy EHR.

Conflict of Interest

None.

Acknowledgments

The authors would like to thank Robert DeLorenzo, MD, for providing guidance on the Institutional Review Board and manuscript and Joel Michalek, PhD, and Mr. Brian Hernandez for providing statistical analysis and support.

Note

The views expressed herein are those of the authors and do not reflect the official policy or position of San Antonio Military Medical Center, the U.S. Army Medical Department, the U.S. Army Office of the Surgeon General, the Department of the Army, the United States Air Force, Department of Defense, or the U.S. Government.

Authors' Contributions

K.K., J.Q., Y.F., and C.S. conceived the study. J.Q., Y.F., V.R., and T.I. designed and wrote the software. K.K., J.Q., V.R., and T.I. performed the data collection. K.K. and D.F. conducted the article search and drafted the article. All authors contributed substantially to article revision. K.K. takes responsibility of the work as a whole.

Protection of Human and Animal Subjects

This study was reviewed and approved by the University of Texas Health at San Antonio Institutional Review Board.

• References

• 1 Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff (Millwood) 2013; 32 (01) 63-68
• 2 Wears RL. Health information technology and victory. Ann Emerg Med 2015; 65 (02) 143-145
• 3 Mohan MK, Bishop RO, Mallows JL. Effect of an electronic medical record information system on emergency department performance. Med J Aust 2013; 198 (04) 201-204
• 4 Adler-Milstein J, DesRoches CM, Kralovec P. , et al. Electronic health record adoption in US hospitals: progress continues, but challenges persist. Health Aff (Millwood) 2015; 34 (12) 2174-2180
• 5 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; 11 (02) 104-112
• 6 Sopan A, Plaisant C, Powsner S, Shneiderman B. Reducing wrong patient selection errors: exploring the design space of user interface techniques. AMIA Annu Symp Proc 2014; 2014: 1056-1065
• 7 Farley HL, Baumlin KM, Hamedani AG. , et al. Quality and safety implications of emergency department information systems. Ann Emerg Med 2013; 62 (04) 399-407
• 8 Handel DA, Wears RL, Nathanson LA, Pines JM. Using information technology to improve the quality and safety of emergency care. Acad Emerg Med 2011; 18 (06) e45-e51
• 9 Hill Jr RG, Sears LM, Melanson SW. 4000 clicks: a productivity analysis of electronic medical records in a community hospital ED. Am J Emerg Med 2013; 31 (11) 1591-1594
• 10 Kamel Boulos MN, Berry G. Real-time locating systems (RTLS) in healthcare: a condensed primer. Int J Health Geogr 2012; 11 (01) 25
• 11 Okoniewska B, Graham A, Gavrilova M. , et al; Ward of the 21st Century team. Multidimensional evaluation of a radio frequency identification wi-fi location tracking system in an acute-care hospital setting. J Am Med Inform Assoc 2012; 19 (04) 674-679
• 12 Ebrahimzadeh F, Nabovati E, Hasibian MR, Eslami S. Evaluation of the effects of radio-frequency identification technology on patient tracking in hospitals: a systematic review. J Patient Saf 2017
• 13 Ohashi K, Ota S, Ohno-Machado L, Tanaka H. Smart medical environment at the point of care: auto-tracking clinical interventions at the bed side using RFID technology. Comput Biol Med 2010; 40 (06) 545-554
• 14 Shirehjini AA, Yassine A, Shirmohammadi S. Equipment location in hospitals using RFID-based positioning system. IEEE Trans Inf Technol Biomed 2012; 16 (06) 1058-1069
• 15 Singman EL, Haberman CV, Appelbaum J. , et al. Electronic tracking of patients in an outpatient ophthalmology clinic to improve efficient flow: a feasibility analysis and benchmarking study. Qual Manag Health Care 2015; 24 (04) 190-199
• 16 Stahl JE, Drew MA, Kimball AB. Patient-clinician concordance, face-time and access. Int J Health Care Qual Assur 2014; 27 (08) 664-671
• 17 Stübig T, Suero E, Zeckey C. , et al. Improvement in the workflow efficiency of treating non-emergency outpatients by using a WLAN-based real-time location system in a level I trauma center. J Am Med Inform Assoc 2013; 20 (06) 1132-1136
• 18 Stahl JE, Drew MA, Leone D, Crowley RS. Measuring process change in primary care using real-time location systems: feasibility and the results of a natural experiment. Technol Health Care 2011; 19 (06) 415-421
• 19 Stahl JE, Drew MA, Kimball AB. Real-time location systems, normative messaging and modifying clinician behavior: a pilot study. Health Syst (Basingstoke) 2014; 3 (03) 165-172
• 20 Ajami S, Bagheri-Tadi T. Barriers for adopting electronic health records (EHRs) by physicians. Acta Inform Med 2013; 21 (02) 129-134
• 21 Bowman S. Impact of electronic health record systems on information integrity: quality and safety implications. Perspect Health Inf Manag 2013; 10: 1c
• 22 Carspecken CW, Sharek PJ, Longhurst C, Pageler NM. A clinical case of electronic health record drug alert fatigue: consequences for patient outcome. Pediatrics 2013; 131 (06) e1970-e1973
• 23 Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J Am Med Inform Assoc 2012; 19 (e1): e145-e148
• 24 Virginio Jr LA, Ricarte IL. Identification of patient safety risks associated with electronic health records: a software quality perspective. Stud Health Technol Inform 2015; 216: 55-59
• 25 Rosenbaum L. Transitional chaos or enduring harm? The EHR and the disruption of medicine. N Engl J Med 2015; 373 (17) 1585-1588
• 26 Bishop RO, Patrick J, Besiso A. Efficiency achievements from a user-developed real-time modifiable clinical information system. Ann Emerg Med 2015; 65 (02) 133-142
• 27 Han YY, Carcillo JA, Venkataraman ST. , et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; 116 (06) 1506-1512