Keywords
electronic health record - telemedicine - COVID-19 pandemic - burden
Background and Significance
Background and Significance
In the spring of 2020, New York City, including the Montefiore Health System (MHS), had its first surge of coronavirus disease 2019 (COVID-19) patients, requiring a rapid and large-scale response from hospital systems. Both inpatient and outpatient operations were restructured to address the increased patient needs, as well as to mitigate the spread of the virus in such a dense population. Although some changes were temporary, such as the cancellation of elective procedures and redeployment of physicians to inpatient services, other changes, such as the widespread implementation of telemedicine, remained in place even after the surge ended.[1]
[2]
Physician experience and burden as affected by these unprecedented changes have understandably been a focus of recent studies.[3]
[4]
[5]
[6]
[7] Surveys of physicians have sought to provide insight into how these changes may impact work-flow and how that may affect burnout rates.[6]
[7] However, with the heterogeneity of the target population, given different specialties, practice locations, and responsibilities, as well as the often rapidly changing clinical environment, surveys may be too burdensome and lack the granularity needed to fully understand how changes may be affecting physicians. A few limited studies have examined the feasibility of electronic health record (EHR)-generated metrics to reflect the changes in physician usage patterns surrounding the time of the pandemic.[3]
[8] One study, limited to a cancer center, utilized such metrics and found that during the initial COVID peak, physicians were spending more time in the EHR, on documentation, and in their inbox, although these metrics returned to near baseline in the follow-up period 3 months later.[9] Our prior study compared metrics derived from provider action logs from two time periods (August 2019 and May 2020) to examine differences due to factors relating to the pandemic and the implementation of telemedicine.[3] Our preliminary findings identified not only differences in EHR usage between the two periods, but also other factors such as individual characteristics (gender, years since graduation) and specialty category that may also contribute to these differences. Following these initial observations, this study seeks to further this exploration by expanding the number and types of metrics evaluated and by broadly including all active physicians who practiced in any of the outpatient clinics in MHS, which includes primary care, subspecialty care, and surgical specialty care clinics.
With the growing presence and impact of EHR and other technologies on physicians and burnout, there is a need for a scalable and nonintrusive method to measure physician usage and experience. Understanding the impact of both systemic-level factors and user-level characteristics would be imperative to ongoing efforts to support and optimize physician experience and health care delivery.
Objectives
The objectives of this study were to explore physician EHR usage patterns during the COVID-19 pandemic through (1) examining variation in EHR-generated metrics across three time periods that correlate with preimplementation of telemedicine and pre-COVID (August 2019), the initial telemedicine postimplementation period (May 2020), and a follow-up period where both telemedicine and in-person visits were utilized (July 2020), and (2) determining the association of gender, specialty category, and years since graduation with these variations in user-level outcome metrics among the three time periods.
Methods
We used Epic Signal, a tool within Epic, the EHR used across MHS since 2014, that generates documentation data that have been previously found useful to studying physician usage patterns.[3]
[10] We included all attending physicians who were active in the EHR during three 1-month periods. August 2019 was selected as the baseline pretelemedicine implementation period. Telemedicine implementation occurred during the initial surge of patients with COVID-19 in March of 2020. Therefore, May 2020 was selected as the postimplementation period and July 2020 as the follow-up and return-to-normal operation time interval (with telemedicine and in-person clinical visits) to evaluate for persistence of changes.[1]
Physician characteristic data were collected to evaluate for associations between gender, specialty type (primary care vs. subspecialty vs. surgery), and years since graduation of their degree-granting program with our 11 metrics of interest. We selected metrics using the Signal web-based dashboard, focusing on metrics displayed under “Overview,” “In Basket,” “Notes and Letters,” and “Workload” metrics as we hypothesized that these would be impacted by the pandemic and subsequent shift from in-person to telehealth to hybrid. Patient volume was assessed using number of appointments per day (number of appointments/number of scheduled days) and number of aggregate messages received per day. Signal has measures for “efficiency” developed by Epic that have been used by prior studies to study changes in physician workflow and usage patterns.[11]
[12]
[13] We included these with the understanding that they may not correlate perfectly with actual efficiency: (1) same day visit closure rate; (2) Physician Efficiency Profile (PEP) score, which compares time the user spent in the system and expected time in the system based on workload, calculated using all other users in the system, with a higher score indicating less time spent compared with expected[11]; (3) proficiency score (measures frequency of use of embedded EHR efficiency tools)[12]
[13]; (4) time (minutes) spent in notes per appointment; (5) turnaround time (days to respond to in-basket messages); and (6) time (minutes) spent in in-basket per day. When selecting which metrics to use to assess for physician burden, we used metrics that evaluate time spent in the EHR outside working hours. Studies have demonstrated associations between both self-reported[14]
[15]
[16] and objective[17] measures of time spent in the EHR with physician burnout as measured by validated surveys such as the Maslach Burnout Inventory survey and the American Medical Association mini-Z measurement. We evaluated burden using the following metrics: (1) time (average minutes) spent in EHR outside of 7 a.m. to 7 p.m.; (2) time (average minutes) spent in EHR on unscheduled days (days with no scheduled patients); and (3) pajama time (average minutes spent in the EHR outside of 7 a.m. to 5:30 p.m. on weekdays and outside unscheduled time on weekends).
Descriptive statistics were generated for all 11 metrics across the three time periods among all physicians, by specialty group, and by gender. The Wilcoxon signed rank test[18] was used to compare the differences in metrics between both postimplementation periods and pre-COVID for the overall sample. The Kruskal–Wallis test or ANOVA (analysis of variance) was used to compare the differences among the three specialty types within each time period, depending on the distribution of the data, and the Mann–Whitney U test was used to compare the differences between gender within each time period.[19] The Spearman rank correlation coefficient[20] was used to assess the associations between four selected metrics (appointments per day, same day visit closure rate, time spent outside of 7 a.m.–7 p.m., and PEP score) with years since graduation. p-Values were not adjusted for multiple testing.
Multivariable analyses using linear regression models were performed for the four selected metrics to evaluate the independent effects of physician characteristics including gender, specialty category, and years since graduation in May 2020 and July 2020 after adjusting for prepandemic levels.
Results
We analyzed data from 495 physicians across 55 specialties and subspecialties. In total, 53% (n = 263) of physicians were female; 36% (n = 176) of physicians were in primary care; 53% (n = 263) in subspecialty care; and 11% (n = 56) in surgical specialties. The average year since graduation was 23 years.
Outcome Metrics Across Time for All Physicians
There was significant variation in outcome metrics across the three time periods ([Fig. 1]). Patient volume in the initial postimplementation period of May 2020 was significantly decreased compared with the volume in August 2019 ([Table 1]), as measured by both number of appointments per day (7.6 compared with 11.6, p < 0.01) and aggregate messages (decreased to 10.0 from 14.7, p < 0.01). However, while the median number of appointments per day in July 2020 also remained significantly less than prepandemic numbers (9.6 compared with 11.6, p < 0.01), aggregate messages increased significantly from 14.7 to 16.6 (p < 0.01).
Table 1
Descriptive statistics of outcome metrics for pre- and post-COVID across all physicians (N = 495)
|
Time period
|
p-Value
(May vs. August)
|
p-Value (July vs. August)
|
|
August 2019
|
May 2020
|
July 2020
|
|
Appointments per day
|
11.6 (8.0–17.3)
|
7.6 (5.3–10.7)
|
9.6 (6.6–13.1)
|
<0.01
|
<0.01
|
|
Aggregate messages
|
14.7 (9.4–23.3)
|
10.0 (5.9–17.8)
|
16.6 (10.1–27.3)
|
<0.01
|
<0.01
|
|
Same day visit closure rate
|
0.7 (0.5–0.9)
|
0.9 (0.7–1.0)
|
0.8 (0.5–1.0)
|
<0.01
|
<0.01
|
|
PEP score
|
4.8 (4.0–5.6)
|
4.9 (3.9–5.7)
|
4.8 (3.9–5.6)
|
0.92
|
0.97
|
|
Proficiency score
|
4.0 (2.8–5.7)
|
4.0 (2.8–5.9)
|
3.9 (2.5–5.9)
|
0.017
|
<0.01
|
|
Time in notes
|
6.2 (3.8–9.7)
|
5.9 (3.8–9.3)
|
5.6 (3.5–8.9)
|
0.22
|
0.03
|
|
Turnaround time
|
4.0 (1.9–8.7)
|
3.0 (1.3–7.6)
|
3.2 (1.4–7.3)
|
0.11
|
0.01
|
|
Time in in-basket
|
10.3 (5.1–17.5)
|
7.9 (3.5–15.2)
|
9.6 (4.9–18.1)
|
<0.01
|
0.49
|
|
Time outside of 7 a.m.—7 p.m.
|
19.3 (6.7–39.1)
|
11.7 (4.7–25.2)
|
16.2 (6.0–31.9)
|
<0.01
|
<0.01
|
|
Time on unscheduled days
|
36.5 (20.3–64.4)
|
24.5 (13.3–42.5)
|
31.5 (16.5–55.6)
|
<0.01
|
<0.01
|
|
Pajama time
|
28.6 (12.0–60.5)
|
15.1 (5.8–35.9)
|
25.0 (7.6–46.8)
|
<0.01
|
<0.01
|
Abbreviation: PEP, Physician Efficiency Profile.
Note: Analysis done using Wilcoxon signed rank test. Median values reported with interquartile ranges.
Fig. 1 Changes in aggregate messages, appointments per day, same day visit closure rate, and time outside 7 a.m. to 7 p.m. across three time periods.
Regarding “efficiency” metrics, same day visit closure rates increased significantly in both postimplementation periods, from 0.7 in August 2019 to 0.9 in May 2020 and 0.8 in July 2020. There was a significant decrease in time in notes (5.6 from 6.2, p = 0.03) and turnaround time (3.2 from 4.0, p = 0.01) in July 2020 compared with August 2019 although not in May 2020. Time in in-basket was initially decreased in May 2020 (7.9 vs. 10.3, p < 0.01) although this difference was no longer significant by July 2020. The PEP score was not significantly different between all three time periods.
Time spent in the EHR outside working hours all significantly decreased during both postimplementation periods compared with August 2019 (p < 0.01).
Outcome Metrics by Gender
Differences between male and female physicians remained consistent across all three periods ([Table 2]). Male physicians had significantly more appointments per day than female physicians (p < 0.04). They also had higher same day visit closure rates (p < 0.01), higher PEP scores (p < 0.01), and spent less time in in-basket (p < 0.01) or in notes (p < 0.01), but had no significant difference in proficiency score (p = 0.19–0.63) or turnaround time (p = 0.07–0.23). Male physicians spent significantly less time in the EHR during pajama time (p < 0.01), unscheduled days (p < 0.01), and outside 7 a.m. to 7 p.m. (p < 0.01).
Table 2
Descriptive statistics of outcome metrics (by gender)
|
August 2019
|
|
Male (n = 232)
|
Female (n = 263)
|
p-Value
|
|
Appointments per day
|
12.7 (8.7–19.4)
|
10.6 (7.4–15.8)
|
<0.01
|
|
Aggregate messages
|
15.1 (10.0–23.5)
|
14.6 (8.9–22.6)
|
0.39
|
|
Same day visit closure rate
|
0.8 (0.5–1.0)
|
0.7 (0.4–0.9)
|
<0.01
|
|
PEP score
|
5.0 (4.1–5.9)
|
4.7 (3.9–5.4)
|
<0.01
|
|
Proficiency score
|
4.0 (2.8–5.7)
|
3.9 (2.8–5.6)
|
0.38
|
|
Time in notes
|
5.1 (2.9–7.9)
|
7.1 (4.8–11.9)
|
<0.01
|
|
Turnaround time
|
3.6 (1.8–8.8)
|
4.3 (2.0–8.7)
|
0.23
|
|
Time in in-basket
|
9.2 (4.4–16.5)
|
11.0 (5.9–18.1)
|
0.01
|
|
Time outside of 7 a.m.–7 p.m.
|
13.2 (6.0–30.6)
|
25.6 (8.2–42.6)
|
<0.01
|
|
Time on unscheduled days
|
32.0 (17.7–54.5)
|
42.6 (24.0–71.1)
|
<0.01
|
|
Pajama time
|
22.4 (10.0–55.3)
|
35.2 (14.3–63.4)
|
<0.01
|
|
May 2020
|
|
Male (n = 232)
|
Female (n = 263)
|
p-Value
|
|
Appointments per day
|
8.2 (5.7–10.8)
|
7.2 (5.0–10.4)
|
0.04
|
|
Aggregate messages
|
9.5 (5.5–18.2)
|
10.3 (6.1–17.7)
|
0.16
|
|
Same day visit closure rate
|
0.9 (0.8–1.0)
|
0.9 (0.6–1.0)
|
<0.01
|
|
PEP score
|
5.2 (4.4–6.0)
|
4.7 (3.5–5.5)
|
<0.01
|
|
Proficiency score
|
3.9 (2.7–5.9)
|
4.0 (2.8–5.8)
|
0.63
|
|
Time in notes
|
4.9 (2.9–7.3)
|
7.0 (4.8–11.5)
|
<0.01
|
|
Turnaround time
|
2.5 (1.0–8.5)
|
3.3 (1.6–7.2)
|
0.07
|
|
Time in in-basket
|
6.1 (2.7–11.8)
|
9.8 (4.3–17.2)
|
<0.01
|
|
Time outside of 7 a.m.–7 p.m.
|
10.0 (3.6–20.9)
|
13.8 (6.0–27.8)
|
<0.01
|
|
Time on unscheduled days
|
19.6 (12.0–32.9)
|
31.3 (16.4–47.9)
|
<0.01
|
|
Pajama time
|
11.6 (4.7–26.9)
|
19.7 (7.2–40.2)
|
<0.01
|
|
July 2020
|
|
Male (n = 232)
|
Female (n = 263)
|
p-Value
|
|
Appointments per day
|
10.5 (7.3–14.2)
|
9.0 (6.0–11.9)
|
<0.01
|
|
Aggregate messages
|
16.0 (10.6–28.5)
|
16.6 (9.7–26.7)
|
0.93
|
|
Same day visit closure rate
|
0.9 (0.6–1.0)
|
0.7 (0.5–0.9)
|
<0.01
|
|
PEP score
|
5.1 (4.1–6.1)
|
4.7 (3.8–5.5)
|
<0.01
|
|
Proficiency score
|
4.0 (2.6–6.0)
|
3.9 (2.3–5.6)
|
0.19
|
|
Time in notes
|
4.6 (2.6–7.4)
|
6.7 (4.4–10.7)
|
<0.01
|
|
Turnaround time
|
2.7 (1.2–7.5)
|
3.4 (1.8–7.2)
|
0.11
|
|
Time in in-basket
|
7.4 (4.2–16.1)
|
11.3 (6.2–20.0)
|
<0.01
|
|
Time outside of 7 a.m.–7 p.m.
|
13.2 (4.6–26.5)
|
20.7 (7.2–36.7)
|
<0.01
|
|
Time on unscheduled days
|
26.0 (12.8–43.9)
|
39.3 (20.9–61.1)
|
<0.01
|
|
Pajama time
|
20.1 (6.8–39.2)
|
30.2 (8.8–55.9)
|
<0.01
|
Abbreviation: PEP, Physician Efficiency Profile.
Note: Analysis done using the Mann–Whitney U test. Median values reported with interquartile values for variables with skewed distributions. Mean values with standard deviations reported for normally distributed variables.
Outcome Metrics by Specialty Category
In August 2019, physicians in surgical specialties had the most median appointments per day (p < 0.01, [Table 3]); however, during May 2020, they had significantly less appointments than primary care or specialty care (p < 0.01). By July 2020, surgery once again had significantly more appointments per day (p < 0.01). Primary care had the most aggregate messages during all three time periods (p < 0.01).
Table 3
Descriptive statistics of outcome metrics (by specialty category)
|
August 2019
|
|
Primary care
|
Specialty care
|
Surgery
|
p-Value
|
|
Appointments per day
|
12.2 (8.3–17.0)
|
10.1 (7.3–15.2)
|
24.7 (16.9–32.5)
|
<0.01
|
|
Aggregate messages
|
19.9 (12.7–36.8)
|
12.3 (8.3–19.3)
|
14.7 (9.9–19.6)
|
<0.01
|
|
Same day visit closure rate
|
0.7 (0.4–0.9)
|
0.8 (0.5–0.9)
|
0.9 (0.6–1.0)
|
0.01
|
|
PEP score
|
4.7 (3.8–5.6)
|
4.8 (4.0–5.6)
|
5.2 (4.6–5.7)
|
0.07
|
|
Proficiency score
|
4.3 (3.6–5.9)
|
3.7 (2.2–5.4)
|
3.9 (2.2–6.2)
|
<0.01
|
|
Time in notes
|
7.0 (5.3–11.9)
|
6.5 (4.0–9.6)
|
1.5 (0.4–2.9)
|
<0.01
|
|
Turnaround time
|
4.1 (2.0–8.4)
|
3.9 (1.8–8.7)
|
4.0 (2.1–12.8)
|
0.68
|
|
Time in in-basket
|
17.3 (10.6–26.8)
|
8.7 (4.7–13.5)
|
4.5 (2.4–7.6)
|
<0.01
|
|
Time outside of 7 a.m.–7 p.m.
|
31.8 (14.3–49.5)
|
13.8 (5.9–32.3)
|
6.4 (2.6–17.1)
|
<0.01
|
|
Time on unscheduled days
|
51.6 (30.2–86.0)
|
34.9 (20.3–58.4)
|
20.3 (14.3–33.1)
|
<0.01
|
|
Pajama time
|
52.1 (21.3–88.1)
|
23.9 (9.2–53.1)
|
15.5 (7.3–27.4)
|
<0.01
|
|
May 2020
|
|
Primary care
|
Specialty care
|
Surgery
|
p-Value
|
|
Appointments per day
|
9.1 (5.6–12.7)
|
7.1 (5.2–9.4)
|
6.8 (4.7–10.7)
|
<0.01
|
|
Aggregate messages
|
15.6 (8.6–30.3)
|
8.9 (5.8–15.0)
|
5.0 (3.6–8.1)
|
<0.01
|
|
Same day visit closure rate
|
0.9 (0.7–1.0)
|
0.9 (0.7–1.0)
|
0.9 (0.8–1.0)
|
0.09
|
|
PEP score
|
4.6 (1.86)[a]
|
4.8 (1.6)
|
5.3 (0.99)
|
0.03
|
|
Proficiency score
|
4.3 (3.3–6.0)
|
3.7 (2.6–5.7)
|
4.0 (2.5–6.3)
|
0.10
|
|
Time in notes
|
6.7 (4.8–10.8)
|
6.1 (4.0–9.3)
|
1.8 (0.8–4.3)
|
<0.01
|
|
Turnaround time
|
3.0 (1.5–7.9)
|
2.8 (1.1–6.7)
|
3.5 (1.4–15.0)
|
0.16
|
|
Time in in-basket
|
14.3 (8.6–23.8)
|
6.1 (3.1–10.8)
|
2.7 (1.5–4.8)
|
<0.01
|
|
Time outside of 7 a.m.–7 p.m.
|
15.4 (6.7–32.2)
|
11.7 (5.0–23.1)
|
5.1 (2.0–12.7)
|
<0.01
|
|
Time on unscheduled days
|
29.7 (14.1–51.5)
|
26.5 (14.5–40.5)
|
14.1 (8.8–19.7)
|
<0.01
|
|
Pajama time
|
17.2 (7.1–40.3)
|
18.0 (6.2–36.3)
|
5.5 (3.0–11.9)
|
<0.01
|
|
July 2020
|
|
Primary care
|
Specialty care
|
Surgery
|
p-Value
|
|
Appointments per day
|
9.6 (6.3–12.7)
|
9.0 (6.5–11.8)
|
14.9 (11.6–21.3)
|
<0.01
|
|
Aggregate messages
|
24.2 (13.0–42.9)
|
14.0 (8.8–21.3)
|
13.6 (8.2–17.6)
|
<0.01
|
|
Same day visit closure rate
|
0.8 (0.5–1.0)
|
0.8 (0.5–1.0)
|
0.9 (0.7–1.0)
|
0.05
|
|
PEP score
|
4.7 (3.6–5.5)
|
4.8 (3.9–5.6)
|
5.1 (4.7–5.8)
|
0.02
|
|
Proficiency score
|
4.2 (3.0–6.0)
|
3.7 (2.3–5.6)
|
4.6 (2.1–6.2)
|
0.08
|
|
Time in notes
|
7.2 (4.7–10.6)
|
5.8 (3.8–8.9)
|
1.7 (0.7–3.4)
|
<0.01
|
|
Turnaround time
|
3.7 (1.7–6.8)
|
3.0 (1.3–7.0)
|
3.2 (1.3–9.3)
|
0.32
|
|
Time in in-basket
|
17.7 (9.7–27.9)
|
7.8 (4.4–14.3)
|
3.8 (2.2–6.4)
|
<0.01
|
|
Time outside of 7 a.m.–7 p.m.
|
27.6 (11.6–41.3)
|
12.8 (4.9–26.5)
|
6.9 (2.3–18.4)
|
<0.01
|
|
Time on unscheduled days
|
40.9 (21.4–77.9)
|
32.7 (15.9–52.5)
|
17.2 (11.5–23.8)
|
<0.01
|
|
Pajama time
|
31.6 (11.1–65.8)
|
26.0 (7.3–45.0)
|
11.1 (2.1–25.2)
|
<0.01
|
Abbreviation: PEP, Physician Efficiency Profile.
a Median (IQR) given for continuous outcomes that are skewed among specialty groups. Mean (SD) given for those normally distributed.
Note: Analysis done using Kruskal–Wallis or ANOVA tests. Median values reported with interquartile values for variables with skewed distributions. Mean values with standard deviation reported for normally distributed variables.
The PEP score was not significantly different between specialty categories in the preimplementation period (p = 0.07), but primary care physicians had significantly lower PEP scores compared with the other specialties in May 2020 (p = 0.03) and July 2020 (p = 0.02). The same day visit closure rate was significantly different in August 2019 (p = 0.01) and July 2020 (p = 0.05), but not in May 2020 (p = 0.09).
In all three time periods, primary care physicians spent significantly more time in the EHR during unscheduled days (p < 0.01) and outside 7 a.m. to 7 p.m. ([Fig. 2]). They also spent significantly more time in the EHR during pajama time in August 2019 (p < 0.01) and July 2020; however, in May 2020 (p < 0.01), specialty care spent more time during pajama time (p < 0.01).
Fig. 2 Differences in time spent in EHR outside 7 a.m.to 7 p.m. by specialty category and time period. HER, electronic health record.
Outcome Metrics and Years since Graduation
Physicians with more years since graduation had significantly higher same day visit closure rates in August 2019 ([Table 4], r = 0.09; p = 0.05) and more appointments per day in May 2020 (r = 0.14, p < 0.01).
Table 4
Correlation between outcome metrics and years since graduation
|
Years since graduation
|
|
r
|
p-Value
|
|
Appointments per day (Aug.)
|
0.08
|
0.06
|
|
Appointments per day (May)
|
0.14
|
<0.01
|
|
Same day visit closure rate (Aug.)
|
0.09
|
0.05
|
|
Same day visit closure rate (May)
|
0.06
|
0.19
|
|
PEP score (Aug.)
|
0.07
|
0.15
|
|
PEP score (May)
|
−0.03
|
0.44
|
|
Time outside of 7 a.m.–7 p.m. (Aug.)
|
0.06
|
0.16
|
|
Time outside of 7 a.m.–7 p.m. (May)
|
0.02
|
0.68
|
Abbreviation: PEP, Physician Efficiency Profile.
Note: Analysis done using Spearman rank correlation coefficient.
Multivariable Analyses
Specialty category was independently associated with number of appointments across all three time periods (p < 0.01, [Table 5]). After adjusting for gender and years since graduation, surgical physicians had significantly fewer appointments compared with primary care in May 2020 (p < 0.01) and specialty care in both May 2020 (p < 0.01) and July 2020 (p = 0.03). Gender was not independently associated with appointments per day (p = 0.08–0.30). Years since graduation was predictive of this metric in May 2020 (p = 0.04).
Table 5
Multivariate analyses of provider characteristics on select outcome metrics
|
May 2020 vs. Aug. 2019
|
July 2020 vs. Aug. 2019
|
|
Appointments per day
|
Coeff.
[95% interval]
|
p-Value
|
Coeff.
[95% interval]
|
p-Value
|
|
Female
|
−0.32 [−0.934, 0.29]
|
0.30
|
−0.52 [−1.11, 0.07]
|
0.08
|
|
Primary care[a]
|
5.69 [4.55, 6.84]
|
<0.01
|
0.65 [−0.45, 1.74]
|
0.25
|
|
Specialty care[a]
|
4.01 [2.90, 5.13]
|
<0.01
|
1.20 [0.13, 2.26]
|
0.03
|
|
Years since graduation
|
0.03 [0.001, 0.05]
|
0.04
|
−0.002 [−0.03, 0.02]
|
0.86
|
|
Same day visit closure rate
|
Coeff.
[95% interval]
|
p-Value
|
Coeff.
[95% interval]
|
p-Value
|
|
Female
|
−0.03 [−0.7, 0.00]
|
0.05
|
−0.02 [−0.05, 0.01]
|
0.22
|
|
Primary care[a]
|
0.01 [−0.04, 0.07]
|
0.63
|
−0.00 [−0.06, 0.05]
|
0.90
|
|
Specialty care[a]
|
−0.03 [−0.09, 0.02]
|
0.22
|
−0.05 [−0.11, −0.003]
|
0.04
|
|
Years since graduation
|
0.00 [−0.001, 0.001]
|
0.96
|
0.00 [−0.0004, 0.002]
|
0.22
|
|
PEP score
|
Coeff.
[95% interval]
|
p-Value
|
Coeff.
[95% interval]
|
p-Value
|
|
Female
|
−0.50 [−0.71, −0.28]
|
<0.01
|
−0.23 [−0.42, −0.03]
|
0.02
|
|
Primary care[a]
|
−0.10 [−0.46, 0.25]
|
0.56
|
−0.27 [−0.59, 0.05]
|
0.10
|
|
Specialty care[a]
|
−0.20 [−0.54, 013]
|
0.23
|
−0.29 [−0.59, 0.01]
|
0.06
|
|
Years since graduation
|
−0.01 [−0.02, −0.01]
|
<0.01
|
−0.01 [−0.02, −0.0001]
|
0.05
|
|
Time outside of 7 a.m.–7 p.m.
|
Coeff.
[95% interval]
|
p-Value
|
Coeff.
[95% interval]
|
p-Value
|
|
Female
|
0.67 [−2.21, 3.54]
|
0.65
|
−0.13 [−3.50, 3.23]
|
0.94
|
|
Primary care[a]
|
2.87 [−2.06, 7.81]
|
0.25
|
3.81 [−2.12, 9.74]
|
0.21
|
|
Specialty care[a]
|
2.85 [−1.68, 7.39]
|
0.22
|
0.21 [−5.12, 5.54]
|
0.94
|
|
Years since graduation
|
0.04 [−0.08, 0.15]
|
0.53
|
0.06 [−0.07, 0.19]
|
0.36
|
a Compared with surgery specialties.
For same day visit closure rate, specialty care physicians had a significantly lower rate in July 2020 compared with surgical specialties (p = 0.04), but other variables were not independent predictors. Female physicians and physicians with more years since graduation were significantly associated with lower PEP scores in both May 2020 (p < 0.01) and July 2020 (p = 0.02–0.05).
Gender, specialty category, and years since graduation were not independently predictive of time spent outside of 7 a.m. to 7 p.m.
Discussion
We observed that physician EHR usage patterns varied significantly from August 2019 to May 2020 and July 2020. Patient volume decreased significantly from August 2019 to May 2020, consistent with other studies,[1]
[16] although we observed an increase in aggregate messages in July 2020. The same day visit closure rate, time in notes, turnaround time, and time in in-basket all demonstrated improvement in May or July 2020. We also found that during these postimplementation periods, physicians were spending less time in the EHR outside of 7 a.m. to 7 p.m. on scheduled patient days than the preimplementation period in 2019. Qualitative studies utilizing survey data have had variable results regarding changes in burnout rates among physicians during the pandemic.[4]
[5]
[6]
[7]
[21] While some have found higher rates of burnout in “frontline” specialties hypothesized to be most affected by COVID-19, such as critical care, emergency medicine, and hospital medicine,[6]
[7] one study found within the same specialty (oncology) staff working on a COVID ward had lower rates of burnout compared with staff working on a non-COVID usual ward.[22] However, studies did consistently find that exposure to COVID[5]
[7] was associated with increased risk for burnout.
Regarding physician characteristics, we observed significant differences in both “efficiency” and “burden” metrics based on gender, years since graduation, and specialty category. Differences between female and male physicians may be due to gender-specific differences in regard to childcare and other familial obligations. Prior studies have shown that physician characteristics such as gender and years since graduation have an overall impact on efficiency[23]
[24] and burnout.[6]
[25]
[26]
[27] However, our multivariate analyses found that gender was only a significant factor in PEP score and years since graduation was only an independent predictor in appointments in May 2020 and PEP scores. Neither was an independent predictor for same day visit closure rate or time spent in EHR outside of 7 a.m. to 7 p.m.. Specialty was a significant factor in appointments per day; understandably, as with the cancellation and delay of surgical procedures, surgical specialties saw a larger change in their volume.
Limitations of our study include the timing of the study. With time periods only being a month long and examining only up to July 2020, the pervasive and long-lasting impact of the many systemic, economic, and public health changes that occurred in mid-2020 may not have been fully captured. For example, recent data suggest a significant decrease in telehealth usage by physicians 1 year after the first COVID-19 surge.[28] There was also only a small percentage of physicians from surgical specialties given the decrease in surgical visits and procedures. There are also limitations regarding the metrics. Using vendor-defined metrics may be biased by desires to minimize the degree of burnout associated with EHR use. We should take care when considering their definitions of “efficiency” and normal working hours, which is suggested to be 7 a.m. to 5:30 p.m. or 7 a.m. to 7 p.m. Logistically, Signal incorporates a 5-second time-out rule where the clock stops measuring if no cursor activity is detected after 5 seconds; therefore, the measured time values may not be reflective of actual time. Appointments per day did not take in account whether physicians had full day versus half day clinic sessions, but just whether they were scheduled for clinic appointments. We also did not control for clinical full time equivalent percentages or scheduled patient hours which would have been a more comprehensive value to use in our calculation of time spent in the EHR outside work hours instead of using standardized time periods. Additionally, although time spent working outside normal working hours is known to correlate with burnout, it is clear that there are other contributors to burnout, such as autonomy and lack of staff, which were cited as leading factors during departmental surveys. The gold standard for burnout is quantitative burnout survey data, which have not been directly measured in our study. Due to the exploratory and extensive nature of our analyses, there likely were statistically significant differences that were due to chance alone and additional focused studies are warranted.
In summary, our study has important implications for using EHR-generated reports to monitor trends in physician experience as impacted by key changes and events, as well as how individual characteristics may impact those trends. Health systems are finding it challenging to simultaneously improve productivity and reduce physician burden. Our data demonstrate that easily accessible EHR data can be used to assess productivity and surrogate markers of burden to improve both. Another important aspect of our study is that our approach is easily scalable and less intrusive (as surveys or other user reporting requirements) on already time-constrained physicians, although utilization of existing peri-pandemic departmental burnout surveys would help further validate its use. Incorporation of operational data such as no-show rates and average wait times can help generate an even more accurate model to be correlated with burnout survey data. Repeated studies across different time periods can further help validate this method to quantitatively and continuously assess physician experience with the EHR to identify areas of interest and improvement.
Conclusion
As health systems aim to improve efficiency without increasing burden on physicians, our study demonstrates that utilizing EHR-generated reports may provide a scalable and unintrusive way to monitor trends in physician usage and experience when impacted by a variety of factors, from individual-level characteristics to large-scale systematic changes such as the pandemic and subsequent widespread implementation of telehealth.
Clinical Relevance Statement
Clinical Relevance Statement
This study provides further support that EHR-generated reports may be used as objective measures that vary significantly based on both external factors and provider-level characteristics. Understanding the factors that impact provider experience and usage patterns can help hospital systems monitor the impact of systemic changes and optimize efficiency and provider burnout. Further efforts should be made to understand the correlation of these metrics and how to best utilize them.
Multiple Choice Questions
Multiple Choice Questions
-
Which of the following has been found to correlate with increased physician burnout?
-
Work environments focused on quality and safety
-
Increased time spent at home on work-related tasks
-
Increased levels of autonomy
-
Male physicians
Correct Answer: The correct answer is option b. Studies have found that organizational factors such as increased time spent on work-related tasks at home, lack of autonomy, and increased workloads have correlated with increased reported levels of physician burnout. There has also been survey data that found that female providers have increased rates of burnout.
-
Which of the following significantly increased in the postimplementation follow-up period (July 2020) as compared with the preimplementation period?
Correct Answer: The correct answer is option d. Our results demonstrated that in the follow-up July 2020 period, only aggregate messages had increased to even more than the preimplementation levels, likely related to the delay in care from the initial surge of the pandemic.
