Time Spent on Dedicated Patient Care and Documentation Tasks Before and After the Introduction of a Structured and Standardized Electronic Health Record

Abstract

Background Physicians spend around 35% of their time documenting patient data. They are concerned that adopting a structured and standardized electronic health record (EHR) will lead to more time documenting and less time for patient care, especially during consultations.

Objective This study measures the effect of the introduction of a structured and standardized EHR on documentation time and time for dedicated patient care during outpatient consultations.

Methods We measured physicians' time spent on four task categories during outpatient consultations: documentation, patient care, peer communication, and other activities. Physicians covered various specialties from two university hospitals that jointly implemented a structured and standardized EHR. Preimplementation, one hospital used a legacy-EHR, and one primarily paper-based records. The same physicians were observed 2 to 6 months before and 6 to 8 months after implementation.

We analyzed consultation duration, and percentage of time spent on each task category. Differences in time distribution before and after implementation were tested using multilevel linear regression.

Results We observed 24 physicians (162 hours, 439 consultations). We found no significant difference in consultation duration or number of consultations per hour. In the legacy-EHR center, we found the implementation associated with a significant decrease in time spent on dedicated patient care (−8.5%). In contrast, in the previously paper-based center, we found a significant increase in dedicated time spent on documentation (8.3%) and decrease in time on combined patient care and documentation (−4.6%). The effect on dedicated documentation time significantly differed between centers.

Conclusion Implementation of a structured and standardized EHR was associated with 8.5% decrease in time for dedicated patient care during consultations in one center and 8.3% increase in dedicated documentation time in another center. These results are in line with physicians' concerns that the introduction of a structured and standardized EHR might lead to more documentation burden and less time for dedicated patient care.

Keywords

electronic health record - implementation - healthcare professionals - patient care - documentation time

Background and Significance

During outpatient visits physicians perform many tasks. The most important tasks are patient care, such as talking and listening to the patient, performing a physical examination or performing procedures such as removing sutures, and clinical documentation tasks such as recording a diagnosis, looking up a test result, prescribing medication, or typing a referral letter. Although patient care is regarded the core task of physicians,[1] to enable them to do and account for this work, they need to document and consult clinical data. However, data documentation and processing should not be disproportionally time-consuming. There is a tension between these two categories of tasks: when more time is needed to document and consult patient data, there is less time for patient care. Research suggests that although an electronic health record (EHR) can provide benefits, it could also undermine the development of the patient–physician relationship.[2] Some physicians tend to combine tasks; they talk to the patient while at the same time they document the answers the patient provides. This, however, can lead to patients having the feeling that the physician does not adequately pay attention to them.[3] Physicians want to keep the percentage of work that is spent on documentation tasks as low as possible.[4]

More and more health care organizations want physicians to document data in an EHR in a standardized and structured way at the point of care. In this context, standardized means using standard coding systems (i.e., controlled vocabularies that provide codes for the described concepts) while still offering the option of documenting free text for more complex situations. Standardized and structured data can be reused within the care process as well as for secondary purposes, enabling decision support, generating management and audit reports, doing research, and other reuse cases. This may require that physicians change working processes and document more data themselves, in a more detailed and standardized manner. Modern, advanced EHRs support the user in this structured data entry process by implementing structured data entry forms based on standard information models and terminological systems such as SNOMED CT (Systematized Nomenclature of Medicine–Clinical Terms).[5] However, when implementing an EHR, several factors (such as optimization of medication safety, patient empowerment, and data quality) play a role in the final decision on which system to purchase and how to implement that system. There is a risk that efficiency of documentation by physicians is compromised by these other organizational goals.

Studies show mean consultation times between 14:47 and 17:53 (minutes:seconds),[6] and the percentage of time spent on patient care ranging from 17 to 52.9% and on documentation tasks from 34 to 37% in several settings (primary, secondary, and tertiary health care) using EHRs or paper-based records.[7] [8] In the primary care setting, a study on time utilization before and after the implementation of an EHR has been performed, showing a nonsignificant increase of direct patient care from an average 13:24 to 13:36 minutes.[9] In line with the literature, our previous research, based on self-reported measures, shows that physicians report to spend 37.1% of their time on documentation and would ideally spend 6.1% less time on this task.[4] However, our results also showed that physicians were concerned that the introduction of the structured and standardized EHR would lead to a higher documentation burden.[4] This concern is mainly based on the new processes required to work with a structured and standardized EHR. In the new working process, physicians document data in a structured format rather than in free text. This format can consist of a proliferation of separate fields, radio buttons, and long dropdown lists. This may require the physician more mouse clicks and more time to find the correct field to register the data compared with a single free-text field to document information. It might also mean that physicians that are used to work only with paper-based patient records have more difficulties with the transition to the EHR, than their colleagues that are already used to working with an EHR.

Objective

The primary aim of this study is to compare the documentation burden and time for dedicated patient care before and after the introduction of a new structured and standardized EHR during outpatient consultations. Additionally, we evaluate the changes in time spent on combined patient care and documentation, and the length and number of consultations before and after the introduction of the new EHR. Furthermore, we compare the results between a center that originally used a paper-based patient record and a center that used a legacy-EHR before they jointly implemented the same structured and standardized EHR.

Methods

Application

To measure the time physicians spend on various tasks, we developed an online application that is used by observers to log the start and stop time of each task a physician performs. The application presents the observer buttons for each relevant task a physician can perform. The tasks in the application are based on the research questions of this study and discussions with medical specialists. There are four main categories: documentation, patient care, peer communication, and other activities (everything not related to one of the three former categories). [Fig. 1] shows the interface of the application, and [Table 1] provides a detailed description of all tasks/buttons.

Table 1 All tasks/buttons in our measurement tool with the overarching categories and an explanation on the exact type of task

Category	Button	Explanation
	Consult	Starts when the patient enters the consultation room and ends when the patient leaves the room
	Back-up	Used to send a backup of data to the server
Documentation	Documentation	Main task in this category. Used when the observer cannot discern whether the task falls in one of the four other (more specialized) documentation tasks
	Search	When the physician is looking for or reading information from the patient record
	Input	When the physician is putting information into the patient record. Either writing or typing (depending on the system used)
	Medication	The physician orders medication, or reviews current medication
	Additional tests	The physician orders additional tests, e.g., imaging and laboratory tests
Patient care	Direct patient care	The physician has direct contact with the patient. This can be talking to the patient or his/her family, performing a physical exam, or anything that directly involves the patient
	Indirect patient care	Tasks related to providing care, but not directly concerning the patient, e.g., preparing a small surgical procedure or vaccine, or cleaning a medical device
Peer communication	Telephone	All calls performed with either a mobile phone or a land line
	Discussion colleague	All discussions between the physician and a colleague
Other activities	Not care related	All tasks that are not directly related to documentation, peer communication, or providing care, and thus do not fit in one of the other categories, e.g., preparing lectures, reading literature
	Moving	The physician moves between rooms, e.g., to get a patient from the waiting room
	Breaks	Coffee, lunch, and all other types of breaks (used for breaks between consults)

Fig. 1 Interface of the application used by the observers (translated from Dutch). In this figure, “direct patient care” and “input” are selected to indicate the physician is talking to the patient and documenting patient data in the record at the same time.

Each time a physician starts a task, the observer presses the corresponding button, recording the start time. In light of the complex nature of medicine it is possible to perform (and therefore measure) multiple tasks at the same time, e.g., simultaneously talking to the patient and documenting information in the patient record. When a button is turned on, the color changes to indicate that the specific task is being performed. When the physician completes the task the button is turned off, recording the end time. Each data item was sent to the central database via the Internet. This required a stable Internet connection as was present in the hospitals in our study.

If the observer could not discern the exact type of documentation, for example, when it was not clear whether a task was “input” or “medication,” they were instructed to use the main button in the category, which is “documentation.” A similar procedure is present in the “other tasks” category with two subtasks “moving” and “breaks” within the main task “not care related.”

Observers

The 35 observers in our study are all students of the medical or medical informatics programs of the University of Amsterdam and the Free University of Amsterdam. All observers were familiar with the process of patient care and were fluent in Dutch, as this is the main language in both study centers. All observers were instructed during a training session of approximately 1 hour within 1 week before the actual observations took place. During this training, we explained the study, the method, and the tool they were going to use. All buttons of the tool were discussed in detail during the training and again directly before the actual measurement. In each of the four periods of observations, a different group of observers participated.

During the observations, the observers received a written summary of the buttons and their explanation in Dutch. Additionally, they had the possibility to keep field notes to record unforeseen circumstances during the measurements that might influence the results, such as a power outage or a system breakdown.

Physicians and Setting

We invited physicians to participate by email before the new EHR was implemented. This email was sent to physicians that act as contact persons to and collaborate with the EHR implementation team in all departments of both hospitals. These physicians and colleagues from other function groups worked together with the implementation team to adjust and align workflows and on the configuration of the new EHR. In the invitation, physicians were invited to participate themselves and to forward the email to colleagues. For the measurement after the new EHR implementation only physicians that had already participated in the observations before the implementation were asked to participate again.

The physicians that were observed were medical specialists of two university hospitals in Amsterdam, the Netherlands. Both hospitals have jointly implemented a new EHR in 2015 and 2016. Before the implementation one hospital (center 1) worked with a combination of mainly a legacy-EHR and some paper records, whereas the other hospital (center 2) worked predominantly with paper-based patient records. In both centers, we observed a group of physicians at 6 months (center 1) or 2 months (center 2) before the implementation of the EHR, and at 6 (center 1) or 8 months (center 2) after the implementation. The latter measurement was performed at least half a year after implementation to ensure that the physicians had time to adjust to the new documentation processes and software. In both centers, we observed the same group of physicians before and after the introduction, enabling a paired comparison.

Observations

The study design was submitted to the ethics committee of the Academic Medical Center, Amsterdam, and was exempt from review. The physicians were observed for an entire session of outpatient consultations (morning or afternoon). During the observations, the physician and the patient had the possibility to indicate they did not want the observer to be present in the consultation room in which case the observer would wait outside, and would therefore not be able to log any data during these consultations. The start and stop time of a consultation were always registered (whether the observer was in the room or not), giving the possibility to calculate the number of consultations per session and the mean duration.

Analyses

Consultations lasting less than 2 minutes were removed from the analysis to exclude miscoded consultations. The main analysis is based on three main outcomes (dedicated documentation time, dedicated time for patient care, and combined patient care and documentation tasks). These analyses give an indication of how physicians spend their time during the consultation. All data are reported as median scores with interquartile ranges. All analyses were performed on the observed time during the actual consultations, i.e., the time that the patient was present in the consultation room.

Differences in consultation duration and number of consultations per hour were tested using a Wilcoxon signed rank test. Differences between task duration before and after the EHR implementation and between the two hospitals were tested using multilevel linear regression. For each main outcome, a model was defined where the outcome variable was the percentage of time spent on that group of tasks. The fixed effects of our models were the dummy variable indicating the period in which the new EHR was implemented (with value 0 for the period before implementation and 1 for thereafter), the hospital, and their interaction term. We developed two separate sets of models with each set having one of the centers as the reference center. Inclusion of the interaction term in the model was tested with analysis of variance (ANOVA). In each model, the physician was added as a random effect (clustering observations of each physician). All analyses were performed using R version 3.3.1.[10] A p-value of < 0.05 (corresponding to a 95% confidence interval of parameter coefficients excluding 0) was regarded as significant.

Results

During the measurements before and after the EHR implementation a total of 24 physicians were observed by 35 observers for over 162 hours in which 439 consultations were performed. We excluded 31 consultations (from 470 observed consultations) lasting less than 2 minutes. The observers did not report large unforeseen circumstances that might influence the results. [Table 2] details the observed physicians and the observations. In only 4 of the 439 consultations, the observer was asked to leave the room and hence no time measurements could be made. The median duration of the included consultations and the median number of consultations per hour are shown in [Table 3]. This table also reports the results of the test for differences in consultation duration and number of consultations per hour (using a Wilcoxon signed rank test).

Table 2 Demographics of the observed physicians and details of the observed consultations

	Center 1	Center 2
Total physicians	13	11
Male	11	5
Female	2	6
Specialties
Pediatrics	1	3
Gynecology and obstetrics	2	1
Urology	1	0
Endocrinology	1	0
Pulmonology	2	0
Traumatology	1	0
Orthopedic surgery	0	1
Hematology	0	2
Ophthalmology	0	1
Nephrology	0	1
Otorhinolaryngology	0	1
Neurology	0	1
Oral and maxillofacial medicine	2	0
Vascular surgery	3	0
Observation time
Before implementation	47 h:55 min	33 h:33 min
After implementation	45 h:48 min	34 h:50 min
Number of consultations
Before implementation	122	98
After implementation	120	99
Observer not present during consultation	2	2
Total duration of consultations
Before implementation	28 h:49 min	21 h:37 min
After implementation	28 h:29 min	21 h:50 min

Table 3 Median duration and median number of consultations per hour before and after the implementation

		Before implementation	After implementation	Wilcoxon signed rank test
Center 1	Median duration (IQR) in min	11:51 (07:00–17:48)	12:27 (07:58–18:46)	0.094
	Median consultations per h	2.5	2.4	0.588
Center 2	Median duration (IQR) in min	10:14 (07:35–15:05)	10:09 (06:29–16:32)	0.898
	Median consultations per h	3.0	2.8	0.831

Abbreviation: IQR, interquartile range.

[Table 4] shows an overview of the time distribution by presenting the median percentage of time physicians spent on dedicated patient care, dedicated documentation, and combined patient care and documentation, including the variation indicated by the inter quartile range.

Table 4 Median percentage time with interquartile range of the three main categories of tasks

	Center 1		Center 2
	Before	After	Before	After
Median dedicated patient care % (IQR)	60.5 (43.9–79.4)	49.1 (29.6–75.4)	62.2 (44.4–79.1)	61.4 (46.8–73.7)
Median dedicated documentation % (IQR)	1.5 (0.0–11.9)	5.0 (0.0–16.9)	0.0 (0.0–2.5)	7.5 (0.0–16.3)
Median combined patient care and documentation % (IQR)	18.7 (6.3–37.9)	18.2 (4.6–35.7)	22.4 (10.0–45.8)	18.2 (9.1–29.2)

Abbreviation: IQR, interquartile range.

Note: Reported per center per measurement.

Based on the collected data, we created a multilevel model for each of the following three main outcome measures: dedicated patient care, dedicated documentation, and combined patient care and documentation. The ANOVA analysis of the interaction term of center and EHR implementation showed that the inclusion of the interaction term was only significant for the model of dedicated documentation time. The coefficients of the models, including confidence intervals, are listed in [Table 5]. The results in this table indicate whether physicians spent more or less time on our defined main tasks after the implementation, and whether this difference was significant. A coefficient is considered significant when its 95% confidence interval does not include zero.

Table 5 Coefficients (CIs) of regression models on associations between EHR implementation and time for dedicated patient care, dedicated documentation, and combined patient care and documentation, per center

	Intercept (CI)	EHR Implementation (CI)	Center (CI)	Interaction (CI)
Center 1 (legacy-EHR) as reference category
Time for patient care	59.7 (52.7–66.6)	−8.5 (−14.2 to −2.8)	1.6 (−8.8 to 11.9)	6.2 (−2.2 to 14.6)
Documentation time	8.9 (5.0–12.7)	1.2 (−1.3 to 3.7)	−6.3 (−12.1 to −0.6)	7.1 (3.4–10.8) [a]
Combined patient care and documentation	23.7 (17.0–30.3)	−1.9 (−6.1 to 2.4)	2.7 (−7.2 to 12.6)	−2.7 (−9.0 to 3.5)
Center 2 (paper-based records) as reference category
Time for patient care	61.2 (53.6–68.9)	−2.3 (−8.5 to 3.9)	−1.6 (−11.9 to 8.8)	−6.2 (−14.6 to 2.2)
Documentation time	2.5 (−1.7 to 6.8)	8.3 (5.6–11.0)	6.3 (0.6–12.1)	−7.1 (−10.8 to −3.4) [a]
Combined patient care and documentation	26.4 (19.1–33.7)	−4.6 (−9.2 to −0.02)	−2.7 (−12.6 to 7.2)	2.7 (−3.5 to 9.0)

Abbreviations: ANOVA, analysis of variance; CI, confidence interval; EHR, electronic health record.

Note: Bold cells are significant.

^a Addition of interaction term significant to model (ANOVA).

[Table 3] shows no difference in consultation duration and the number of consultations per hour before and after the implementation of the new EHR in either hospital. [Table 5] shows all differences between task duration before and after the EHR implementation and between the two hospitals using multilevel linear regression. Highlighting the most important results, we see that the EHR implementation was significantly associated with an 8.5% decrease in time for patient care in center 1 (using legacy-EHR previously). This means 8.5 percentage points less time is devoted to patient care relative to the total time of the consultation. We did not find a significant difference in time for patient care at baseline or a significant difference in effect of the EHR implementation between the two centers. For dedicated documentation time as outcome, the EHR implementation was significantly associated with an 8.3% increase in center 2 (previously using paper-based records). There was a significant difference in documentation time between the centers at baseline (6.3% lower in center 2) and the effect of the EHR implementation on documentation time between the centers was significantly different (7.1% higher in center 2). The EHR implementation was significantly associated with a 4.6% decrease in time for combined patient care and documentation in center 2. We did not find a significant difference in combined patient care and documentation time at baseline or a difference in effect of the EHR implementation on combined patient care and documentation time between the centers.

Discussion

The implementation of the structured and standardized EHR was significantly associated with a decrease in dedicated time for patient care in the center that previously used a legacy-EHR and an increase in documentation time in the center that previously used paper-based records. We did not observe a significant difference in consultation duration and the number of consultations per hour before and after the implementation of the new EHR in either hospital.

Although the results were different for the two centers, both are in line with the concern most care providers have when a structured and standardized EHR is implemented, i.e., increase of documentation burden and reduction of patient care. Future research will be needed to provide more evidence in similar and other settings. The different results for the two centers might be explained by their differences at baseline. At baseline, the center which used a legacy-EHR already had a significantly higher documentation time of 6.3% point compared with the center which was previously paper-based. This might explain the significantly different effect of the new EHR implementation on documentation time in both centers. Additionally, we found a statistically significant (albeit perhaps not clinically relevant) association between the EHR implementation and a decreased time used for combined patient care and documentation in the previously paper-based center, indicating that talking and listening to a patient might more easily be combined with documentation when a paper-based record is used than when a structured and standardized EHR is used.

Based on our models, the EHR implementation is associated with an 8.5% decrease in dedicated patient time in the center that previously used a legacy-EHR. This means that on average almost 1 minute of patient care of an 11-minute consult is lost. The 8.3% increase in documentation time in the previously paper-based center amounts to the addition of almost 1 minute extra documentation during an 11-minute consult.

Although our study design cannot prove a cause and effect relation between the EHR implementation and the results we found, we do believe that the implementation, and the accompanying changes in work processes, is the strongest factor that could explain the observed decrease of patient time and increase of documentation time. Other influencing factors might include the level of training provided to the physicians, and personal habits and preferences of the physicians. Physicians might prefer to document during or after the consultation. We tried to account for the latter group of confounders (personal habits) by including a diverse group of physicians in our study and measuring the same group before and after the implementation.

Previous research by Scott et al studying the operational impact of digitized hospital records in an English setting showed mean consultation times from various medical specialties to be between 14:47 and 17:53 minutes.[6] Our own median duration varies between 10:09 and 12:27. The difference might be attributed to the organizational differences between England and the Netherlands and differences in specialties included in both studies. Scott et al included gynecology, pediatrics, vascular surgery, and rheumatology,[6] whereas we included a wider range of specialists.

We measured a total mean documentation time (with and without combining it with patient care) of 31% for center 1 and 26% for center 2 before implementation and 33% for both centers after implementation. In our own previous research in the same university hospitals,[4] we found a self-reported outcome of 37.1% documentation time. This self-reported time was measured before the implementation of the structured and standardized EHR and included the documentation over the entire work day, while in this study we only measured during actual consultations where the patient and physician were in the consultation room together.

The study by Sinsky et al[7] is methodologically most comparable to ours and showed a time allocation of 37% for EHR and desk work, and 52.9% for patient care in ambulatory consultation time. Our study results after the EHR implementation show similar percentages of documentation time (33% for both centers) and dedicated patient time (center 1: 52%, center 2: 60%).

The main strength of our study is the comparison of time spent on dedicated patient care and documentation before and after the implementation of a new structured and standardized EHR. By performing our analyses on data from the same group of physicians before and after the implementation, we enabled an accurate comparison. By using a mix of specialties we attempted to provide a good representation of specialties working in a university hospital.

A limiting factor of our study design is that we only measured the time allocation of physicians during outpatient consultation hours, and only analyzed the time that the patient was present in the consultation room (i.e., the actual consult). This means that we are not able to make any conclusions about a possible shift of documentation tasks to time between consultations, after consultations, or even after working hours. Especially, this documentation time after working hours is an important issue for physicians, as this can extend up to 2 hours into personal time.[7] Not including postconsultation time in our study may cause a significant loss of information. This may distort the overall picture of documentation time of physicians.

Another limitation is the inclusion of time spent on handling telephone calls. The observers cannot reliably determine the topics of these phone calls and therefore the time spent on the telephone cannot with certainty be linked to the patient present in the room at that time. Although the amount of time spent on telephone calls is relatively small, our choice to not subtract this time may lead to some overestimation of the total consultation time.

The main limitation of this study is the small number of included physicians. For pragmatic reasons, it was not feasible to include more physicians in both the before and after implementation phase. However, the number of participants in our study is similar to other studies using comparable methods.[9] To answer our research question, other study designs have been considered but these would have created other drawbacks. Analyzing log data such as used in Tai-Seale et al,[11] for example, might be suitable to indicate changes in documentation time. However, the time used for interaction with the patient cannot be extracted from this kind of data. The results of our study could be influenced by the type of patient that presented itself during the consultations in our study period. New patients might require more documentation work and elderly patients might require more dedicated time for communication. We did not have data on patient characteristics and our number of observations was too small to perform those subgroup analyses. The case mix of patients could be included in future studies.

A potential source of bias might be the distribution of physicians from different specialties in our study. Several specialties were present in both centers; other specialties were present in just one of the two centers. Because we included exactly the same physicians in the before- and after-implementation measurement, this will not influence the effect of the implementation itself. It might, however, influence the difference in effect size between the two centers.

Future research with more available resources might include more physicians enabling more statistical power, and enable subanalysis comparing specialists that rely less or more heavily on detailed patient information such as in Scott et al.[6] Including EHRs of different vendors may show that documentation time varies between different types of EHR.

Conclusion

In conclusion, in our study we found a significant decrease of time for dedicated patient care in one center and a significant increase in dedicated documentation time in the other center, associated with the implementation of a structured and standardized EHR. Different effects might be explained by the baseline situation of a center. We did not observe a significant difference in consultation duration and the number of consultations per hour before and after the implementation. These results are in line with the concern of physicians that the introduction of a new structured and standardized EHR might lead to higher documentation burden and less time for dedicated patient care.

Clinical Relevance Statement

Our results are in line with the concern of physicians that the introduction of a new structured and standardized EHR might lead to higher documentation burden and less time for dedicated patient care. Centers implementing EHRs should be aware of these possible changes and try to minimize the effects these changes can have on the provided patient care.

Multiple Choice Question

This study shows the introduction of a new structured and standardized EHR can lead to:

• An increase in patient satisfaction

• A decreased number of medication errors

• An increased documentation burden

• A decreased documentation burden

  Correct Answer: The correct answer is c, an increased documentation burden. We found that physicians might need more time to do the documentation of patient information during consultations. This can be related to the introduction of a structured and standardized EHR that requires new work processes and more documentation at the point of care.

Conflict of Interest

None.

Acknowledgments

The authors gratefully thank the physicians who allowed us to follow them during part of their daily work. The authors also acknowledge Olav Trauschke for the development of the measurement application and his aid in the phase of the study before the implementation.

Protection of Human and Animal Subjects

The study design was submitted to the ethics committee of the Academic Medical Center, Amsterdam, and was exempt from review.

Funding

None.

• References

• 1 Toll E. A piece of my mind. The cost of technology. JAMA 2012; 307 (23) 2497-2498
• 2 Rathert C, Mittler JN, Banerjee S, McDaniel J. Patient-centered communication in the era of electronic health records: what does the evidence say?. Patient Educ Couns 2017; 100 (01) 50-64
• 3 Lee WW, Alkureishi MA, Ukabiala O. , et al. Patient perceptions of electronic medical record use by faculty and resident physicians: a mixed methods study. J Gen Intern Med 2016; 31 (11) 1315-1322
• 4 Joukes E, De Keizer N, Abu-Hanna A, De Bruijne M, Cornet R. End-User experiences and expectations regarding data registration and reuse before the implementation of a (new) electronic health record: a case study in two university hospitals. Stud Health Technol Inform 2015; 216: 997 . Doi: 10.3233/978-1-61499-564-7-997
• 5 International Health Terminology Standards Development Organisation. SNOMED International. Available at: http://www.snomed.org/ . Accessed May 19, 2017
• 6 Scott PJ, Curley PJ, Williams PB, Linehan IP, Shaha SH. Measuring the operational impact of digitized hospital records: a mixed methods study. BMC Med Inform Decis Mak 2016; 16 (01) 143
• 7 Sinsky C, Colligan L, Li L. , et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
• 8 Tipping MD, Forth VE, O'Leary KJ. , et al. Where did the day go?–a time-motion study of hospitalists. J Hosp Med 2010; 5 (06) 323-328
• 9 Pizziferri L, Kittler AF, Volk LA. , et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform 2005; 38 (03) 176-188
• 10 R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing Austria; 2015. Available at: https://www.R-project.org/ . Accessed May 19, 2017
• 11 Tai-Seale M, Olson CW, Li J. , et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017; 36 (04) 655-662