Associations of Telehealth Care Delivery with Pediatric Health Care Provider Well-Being

Keywords

telemedicine and telehealth - burnout, professional - efficiency - clinical effectiveness - personal satisfaction

Background and Significance

The novel coronavirus disease 19 (COVID-19) pandemic created opportunities and imperatives to introduce and expand telehealth as quickly as possible. Health services across the United States were swiftly transformed to virtual care delivery, often within a few weeks, encouraged by federal and state emergency actions.[1] Prepandemic, 15% of physicians offered telehealth services and 10% of patients had participated in a telehealth encounter. Between spring and summer 2020, 85% of physicians and 46% of patients reported use of telehealth.[2]

Physician stress and burnout, particularly associated with technology use, were issues of considerable interest prepandemic.[3] [4] [5] [6] [7] In the context of a public health crisis that placed extraordinary responsibilities on health care professionals and contributed to deterioration of work–life happiness, clinician well-being has become an even greater concern.[8] [9] As telehealth adoption and availability grows, so does the need to understand providers' perceptions of its benefits, burdens, and barriers.

Few studies have examined the impact of telehealth on provider well-being. Limited evidence from the pre-COVID-19 era on telehealth user and provider wellness and attitudes, largely focused on hospital-based providers, found telehealth decreased on-call travel burdens of physicians and improved their perceptions of teamwork and safety climate.[10] [11] Other studies highlighted the salience of usability and effectiveness for physicians' willingness to adopt telehealth and suggested variability in attitudes between telehealth-experienced and telehealth-inexperienced providers.[12] [13]

This study was guided by a National Academies of Medicine (NAM) conceptual model that links cross-disciplinary clinician resilience and well-being with outcomes for health care professionals, patients, and the broader health system.[14] Like other frameworks of workplace well-being, this model incorporates individual and environmental inputs. It proposes clinician well-being is determined by a balance between demands and resources, with external system factors more influential than internal individual ones. The model consciously situates the health care professional, patient relationship at its center.

The terms “telemedicine” and “telehealth” have been defined in many ways.[15] For this manuscript, we employ the term “telehealth” to refer to synchronous encounters between providers and patients, inclusive of video plus audio, as well as audio only (telephonic) interactions.

Objective

This study sought to describe provider-reported changes in daily work and well-being, accompanying the rapid expansion of telehealth for health care delivery. Provider perceptions of telehealth effectiveness, efficiency, and work–life balance were examined to identify factors associated with improved provider well-being and provider desire for future telehealth.

Methods

Setting

A cross-sectional observational survey study was conducted in fall 2020 by three pediatric health care systems in the United States: Connecticut Children's (CC) Hospital, Nationwide Children's Hospital (NCH), and East Tennessee Children's Hospital (ETCH). These institutions represent two academic tertiary care medical centers (CC: 187 beds and 400,000 annual outpatient visits; and NCH: 625 beds and 1.3 million annual outpatient visits), and one community hospital (ETCH: 152 beds and 120,000 annual outpatient visits). The two larger institutions use an enterprise Epic Electronic Health Record (EHR) platform (Epic Systems Corporation, Verona, Wisconsin, United States) with integrated Zoom video functionality (Zoom Video Communications, San Jose, California, United States); the smallest institution uses a Meditech EHR system (Medical Information Technology Inc., Westwood, Massachusetts, United States) with a nonintegrated Zoom video platform. The study protocol was approved by the Institutional Review Boards (IRB) at CC and NCH and was deemed exempt by the IRB at ETCH.

Survey Design and Sampling

Since most existing telemedicine instruments are intended to assess the patient experience, a brief platform-agnostic provider experience questionnaire was developed by the collaborative research team to inform organizational strategic plans for continuous improvement in telehealth.[16] The questionnaire, which is freely available,[17] drew upon findings of a 2016 American Academy of Pediatrics survey of pediatrician telehealth attitudes and experience.[18] [19] Items were adapted from existing validated clinician technology user and patient telemedicine experience instruments, including the KLAS Electronic Medical Records User Experience Survey, the Telemedicine Satisfaction Questionnaire, and the Telehealth Usability Questionnaire.[20] [21] [22] Novel items were created in domains of telehealth demands, resources, benefits, and barriers. The questionnaire collected demographic information and a self-reported burnout measure previously shown to correlate with the Maslach Burnout Inventory's emotional exhaustion subscale.[23] The survey was pilot tested with 10 providers who suggested edits and confirmed time burden and comprehensibility of the final version.

The survey was deployed using REDCap version 10.6.10 software (Vanderbilt University, Nashville, Tennessee, United States) via e-mail to a list-based population of 947 primary care and subspecialty health providers, excluding trainees, at the three children's hospitals. NCH chose to include psychology providers, while CC and East Tennessee did not. Consent to participate was implied by survey completion. The provider population for this analysis was limited to physicians, advanced practice nurses, physician assistants, and psychologists who reported active engagement in telehealth between March 1, 2020, and October 31, 2020. Statistical analysis was completed in SPSS version 27 (IBM Corporation) and in R 4.0.2 (R Core Team), using the polychor, lavaan, and psych packages.[24] [25] [26]

Provider telehealth-associated well-being was operationalized as the 6-point Likert scale response to “How has the introduction/expansion of telemedicine affected your professional well-being?” with responses of “improved” or “greatly improved” considered positive attributed change. Preference for future telehealth was captured as a 5-point Likert's scale response to “In an ideal world, how much telemedicine would you provide after COVID-19?” Burnout was determined via a five-choice single-item validated measure that has been extensively described in previous literature.[27] [28]

Statistical Analysis

Survey responses were analyzed as standard summary statistics, overall, and by respondent organization to assess for institutional effect. Logistic regression was used to estimate the odds of provider-reported improved telehealth-attributed well-being and the odds of provider-reported desire for future telehealth with demographic factors and responses regarding telehealth resources, demands, benefits, and barriers. Analyses made maximal use of available data by omitting missing values by pairwise deletion. Factor analysis and structural equation modeling were performed to incorporate multiple independent variables, extract latent constructs, and simultaneously analyze interrelationships with multiple outcomes of interest. Well-being, future telehealth preference, and burnout were treated as binary and ordered categorical variables without substantive difference between methods. Narrative comments were examined inductively and coded to generate leading qualitative themes by members of the study team.

Results

Survey response rate was 53.0%. Of 502 respondents, 467 (93.0%) met inclusion criteria by reporting active engagement in telehealth between March and October 2020. Responding providers were not statistically different from nonresponding providers in terms of gender or role. Also, 458 respondents (98.1%) answered at least 9 of 10 designated “core” survey questions and 105 (22.5%) supplied additional narrative comments. Core survey questions included responses to items about telehealth use, change in well-being attributed to telehealth, and information about role, gender, specialty, career stage, and clinical practice setting. Providers volunteered 69 free-text comments regarding telehealth benefits and 57 comments on telehealth barriers and burdens.

Respondent demographics are summarized in [Table 1].

Table 1 Survey respondent demographics

		CC (no.)	ETCH (no.)	NCH (no.)	Overall (no.)	Percent of total (%)
Gender	Female	77	28	220	325	70
	Male	40	10	87	137	29
	Other/not disclosed	2	0	3	5	1
Role	Attending physician	87	26	188	301	66
	Advanced practice provider	26	10	75	111	24
	Psychologist	0	0	47	47	10
Career stage	0–10 years	43	10	131	184	39
	11–20 years	38	9	100	147	32
	21+ years	38	19	79	136	29
Specialty	Medical subspecialist	52	11	157	220	47
	Surgical subspecialist	26	1	41	68	15
	Primary care	7	15	32	54	12
	Hospital care/critical care	12	1	11	24	5
	Psychiatry/psychology	2	0	62	64	14
	Other/unable to be determined	18	8	2	28	6
Practice setting	Outpatient	34	23	141	198	42
	Inpatient	3	0	1	4	1
	ED/urgent care	0	0	1	1	<0.1
	Multiple settings	82	15	167	264	57
Burnout	No symptoms of stress or burnout	25	10	56	91	20
	Stress but no burnout	56	16	149	221	48
	Burnout symptoms	28	9	80	117	25
	Persistent burnout symptoms	6	2	21	29	6
	Complete burnout	0	0	4	4	1
Pre-COVID-19 telehealth experience	Telehealth Naive	108	31	223	362	72

Abbreviations: CC, Connecticut Children's Hospital; COVID-19, novel coronavirus disease 2019; ECTH, East Tennessee Children's Hospital; ED, emergency department; NCH, Nationwide Children's Hospital.

Telehealth Usage

Most survey respondents were telehealth-naïve, having no previous telehealth experience, prior to COVID-19. For all institutions, median provider telehealth experience at baseline was 0% (95% confidence interval [CI]: 0.0–0.0%). Providers experienced dramatic changes in daily work as they rapidly adopted telehealth; median peak telehealth use during the time period between the outbreak of COVID-19 and the survey across all institutions was 77% (95% CI: 75.0–80.0%) of patient interactions. At ETCH, providers did not rely as heavily on telehealth as the larger institutions; median peak usage there was 7.0% (95% CI: 5.0–22.5%). The telehealth transformation at participating institutions was largely focused on ambulatory settings. NCH was not using telehealth in their emergency or inpatient areas at the time of survey. Median peak telehealth use for exclusively hospital-based acute care physicians was 50.0% (95% CI: 2.0–79.0%) of patient encounters.

Provider Well-Being and Desire for Future Telehealth

A total of 378 of 388 (83.1%) providers characterized the shift to telehealth as impactful to well-being. Over one-third of respondents, 35.8% (167/388), described “improved or greatly improved” telehealth-associated well-being. Respondents were 4.77 times as likely (95% CI: 3.44–7.26) to characterize telehealth-related changes in well-being as positive versus negative. Gender, role, career stage, practice setting, and institution were not significantly related to positive well-being ratings. Surgical providers were significantly less likely than behavioral health specialty providers to report improved well-being (odds ratio [OR] = 0.46; 95% CI: 0.22–0.96). Providers with any degree of burnout were significantly less likely to report improved telehealth-attributed well-being (OR = 0.53; 95% CI: 0.34–0.081). Absent burnout symptoms, there was no statistically significant difference in likelihood of improved telehealth-associated well-being between providers who denied and those who endorsed professional stress.

Also, 411 of 467 (95.5%) providers wished to continue telehealth post-COVID-19, with most, (317/467 [67.9%]), preferring to perform at least “some” telehealth. Provider role and specialty were significantly associated with desire for postpandemic telehealth ([Table 2]). Providers with improved telehealth-attributed well-being were 9.96 times as likely (95% CI: 5.41–18.32) to prefer some or more telehealth going forward.

Table 2 Univariable binary logistic regression models of improved well-being and preference for future telehealth

	Telehealth-attributed improved well-being					Preference for some or greater future telehealth
Characteristic	OR	95% CI		p-Value	R 2 (%)	OR	95% CI		p-Value	R 2 (%)
Gender
Female	0.86	0.14	5.19	0.865	0	1.57	0.26	9.53	0.626	1
Male	0.78	0.13	4.85	0.793		1.12	0.18	6.96	0.900
Other/nondisclosed	1 (reference)					1 (reference)
Role
Physician	0.81	0.43	1.54	0.524	2	0.13	0.04	0.43	<0.001	4
APRN	1.12	0.55	2.28	0.755		0.12	0.04	0.42	<0.001
PA	0.16	0.02	1.37	0.094		0.08	0.02	0.43	0.003
Psychologist	1 (reference)					1 (reference)
Specialty
Medical subspecialist	0.88	0.50	1.55	0.656	3	0.16	0.06	0.46	<0.001	7
Surgical subspecialist	0.46	0.22	0.96	0.038		0.07	0.02	0.22	<0.001
Primary care	0.48	0.22	1.05	0.067		0.10	0.03	0.31	<0.001
Hospitalist	0.69	0.26	1.83	0.451		0.07	0.02	0.24	<0.001
Behavioral health specialist	1 (reference)					1 (reference)
Career stage
Late (21+ years)	0.97	0.61	1.56	0.909	1	0.83	0.52	1.33	0.435	0
Mid (11–20 years)	1.36	0.87	2.13	0.183		0.93	0.58	1.49	0.764
Early (<10 years)	1 (reference)					1 (reference)
Practice setting
Ambulatory	0.95	0.65	1.39	0.780	0	1.16	0.78	1.73	0.470	0
Combined	1 (reference)					1 (reference)
Stress and burnout
No stress or burnout	2.33	1.35	4.05	0.003	3	0.77	0.44	1.32	0.337	0
Stress but no burnout	1.74	1.11	2.75	0.016		1.01	0.64	1.57	0.982
> 1 burnout symptoms	1 (reference)					1 (reference)

Abbreviations: APRN, advanced practice nurse; CI, confidence interval; OR, odds ratio; PA, physician assistant.

Telehealth Successes from a Provider Perspective

There was widespread provider consensus of the usefulness of telehealth for clinical care delivery. Furthermore, 60.7% (283/467) described themselves as “mostly” or “completely” able to deliver high quality care through telehealth. Only 4 of 467 respondents (0.9%) reported telehealth is “not at all able” to meet their patient needs. Also, 67.5% (315/467) reported they can “mostly” or “completely” meet patient needs through telehealth. Again, 73.0% (341/467) also felt “mostly” or “completely” able to engage meaningfully with patients via telehealth. Moreover, 84.8% (396/467) do not believe virtual care undermines the provider–patient relationship, reporting telehealth had a positive or net neutral influence on the provider–patient relationship.

Prompted to describe the benefits of telehealth versus in-person care “to you as a provider,” respondents most commonly endorsed ways in which telehealth enables them to expand access to patients (341/467 [73.0%]) and renews their focus on patient-centric care (228/467 [48.8%]). Providers also indicated their own work–life balance (flexibility or control) was a leading benefit 204/467 [43.7%]).

Providers' free text comments described benefits that mapped to themes of improved patient-centered care, improved quality and safety, decreased no-show rates, and personal convenience.

Provider-Identified Telehealth Stressors and Shortcomings

Across the three organizations, provider respondents identified emergent telehealth demands not matched by available telehealth resources as presented in [Fig. 1]. Despite differences in platforms and implementation at participating hospitals, 85% of providers (397/467) named patient and/or provider technology problems (devices, connectivity, sound/image quality, and patient portal issues) as a barrier to effective use of telehealth. Of these, 72.6% (339/467) identified novel technology burdens for providers or patients and 52.7% (245/465) reported inadequate technical support for themselves and their patients.

Fig. 1 Provider-reported telehealth resource adequacy. CC, Connecticut Children's Hospital; ECTH, East Tennessee Children's Hospital; NCH, Nationwide Children's Hospital.

At all organizations, training was a universal unmet need; regardless of institutional curriculum or teaching modality, 56.8% of providers (264/465) rated telehealth training less than “good.”

Further, 43.1% of providers (200/464) noted inadequate clinical support for telehealth encounters, and 32.5% (152/467) described inefficiencies in telehealth clinical processes. Only 9.6% (45/467) of respondents agreed “collaboration” was a benefit of telehealth, suggesting workflows that foster effective teamwork are lacking. More than half of providers (298/467 [63.8%]), identified challenges in obtaining an adequate physical exam via telehealth. Providers also described difficulties determining which patients would be well-served by telehealth.

Aldo, 37.6% of respondents (175/465) reported infrastructure inadequacies including provider space and equipment for telehealth. Video software shortcomings were identified as a top unmet need at one organization.

Providers also offered additional details in their narrative comments regarding leading telehealth stressors which indicted workflow inefficiencies, technical problems, and challenges regarding the appropriateness of telehealth for particular patients. Provider-reported telehealth benefits and barriers/burdens are presented in [Fig. 2].

Fig. 2 Provider-reported telehealth barriers and benefits. CC, Connecticut Children's Hospital; ECTH, East Tennessee Children's Hospital; NCH, Nationwide Children's Hospital.

Factors Associated with Improved Well-Being and Desire to Continue Telehealth

The results of univariate logistic regression modeling of positive telehealth-attributed well-being and preference for substantial future health with provider-reported benefits, barriers, and burdens are presented in [Table 3].

Table 3 Telehealth implementation and use characteristics: univariable logistic regression models of improved well-being and desire for future telehealth

	Telehealth-attributed improved well-being (binary)							Preference for some or greater future telehealth (binary)
	OR	95% CI		p-Value	R 2 (%)	Sensitivity	Specificity	OR	95% CI		p-Value	R 2 (%)	Sensitivity	Specificity
Resources
Adequate training	2.01	1.37	2.95	<0.001	37	0.54	0.63	1.62	1.086	2.423	0.018	1.7	0.47	0.65
Adequate technical support	3.04	2.05	4.51	<0.001	9	0.65	0.62	1.98	1.328	2.955	0.001	3.4	0.53	0.64
Adequate clinical support	2.20	1.47	3.27	<0.001	5	0.69	0.50	2.00	1.347	2.973	0.001	3.6	0.62	0.55
Adequate space and equipment	2.48	1.63	3.77	<0.001	6	0.75	0.45	1.82	1.226	2.713	0.003	2.6	0.67	0.47
Adequate video software	3.04	1.81	5.11	<0.001	6	0.87	0.30	2.79	1.783	4.291	0.001	6	0.82	0.37
Effectiveness
Delivers high quality care	5.66	3.53	9.05	<0.001	17	0.84	0.52	8.95	5.728	13.98	<0.001	28.6	0.77	0.73
Serves patients' needs	6.63	3.87	11.38	<0.001	17	0.89	0.44	7.80	5.025	12.1	<0.001	25.1	0.82	0.63
Engages patients meaningfully	7.16	3.88	13.21	<0.001	16	0.92	0.38	5.77	3.709	8.974	<0.001	17.8	0.85	0.51
Supports provider–patient relationship	7.32	4.79	11.20	<0.001	25	0.65	0.80	4.66	2.834	7.645	<0.001	12.7	0.77	0.73
Benefits
Increased provider efficiency	8.04	5.18	12.49	<0.001	26	0.59	0.85	6.26	3.508	11.16	<0.001	14.7	0.41	0.90
Better work life balance	6.57	4.31	10.00	<0.001	23	0.72	0.72	4.87	3.081	7.688	<0.001	15.1	0.55	0.80
Financial productivity	1.82	1.06	3.15	0.031	1	0.17	0.90	2.31	1.164	4.584	0.017	1.9	0.15	0.93
Expanded patient access	2.41	1.50	3.86	<0.001	4	0.83	0.33	4.28	2.777	6.599	<0.001	12.7	0.83	0.47
Improved patient-centric focus	2.44	1.65	3.60	<0.001	6	0.63	0.59	3.05	2.018	4.603	<0.001	8.6	0.57	0.69
Improved collaboration	2.47	1.32	4.59	0.004	2	0.15	0.93	7.48	2.281	24.56	<0.001	5.5	0.13	0.98
Greater safety	1.46	0.97	2.18	0.068	1	0.37	0.72	1.04	0.684	1.586	0.848	0	0.32	0.69
Barriers and burdens
Workflow inefficiency	0.36	0.23	0.56	<0.001	6	0.40	0.81	0.47	0.316	0.711	<0.001	3.8	0.44	0.73
Physical examination challenges	0.54	0.36	0.80	0.002	3	0.69	0.46	0.38	0.24	0.586	<0.001	5.9	0.78	0.43
Unconfident provider	0.22	0.10	0.51	<0.001	5	0.16	0.96	0.28	0.156	0.492	<0.001	5.7	0.22	0.93
Patient language/culture barriers	0.62	0.41	0.92	0.018	2	0.41	0.70	0.98	0.657	1.468	0.929	0	0.37	0.63
Patient home environment barriers	0.58	0.37	0.89	0.014	2	0.32	0.78	0.86	0.559	1.309	0.472	0.2	0.31	0.73
Privacy concerns	0.39	0.19	0.83	0.015	2	0.13	0.95	0.91	0.48	1.717	0.766	0	0.11	0.90
Technical challenges	0.58	0.31	1.10	0.096	1	0.93	0.12	1.85	0.976	3.521	0.059	1	0.87	0.07
Unreceptive patient	0.69	0.46	1.03	0.067	1	0.40	0.69	1.09	0.724	1.627	0.692	0	0.35	0.63
Reimbursement concerns	1.26	0.77	2.06	0.351	0	0.16	0.80	1.10	0.654	1.837	0.727	0	0.17	0.82
Regulatory concerns	0.06	0.40	0.90	0.013	2	0.40	0.72	1.32	0.871	1.991	0.191	0.5	0.31	0.62

Abbreviations: CI, confidence interval; OR, odds ratio.

To better understand the interplays among telehealth characteristics and outcomes of interest, we performed factor analysis and structural equation modeling to reduce data dimensionality while preserving variable information and accounting for correlations. Six latent factors were initially identified through exploratory factor analysis and three were retained after confirmatory analysis. “Telehealth Patient Effectiveness” includes three variables: (1) the abilities to engage with patients meaningfully, (2) serve patients' needs, and (3) deliver high quality care. “Telehealth Provider Satisfaction” includes three variables that capture providers' opinions of telehealth's impact on their own well-being and the provider-patient relationship, as well as the desire for continued telehealth. “Telehealth Resources” includes five variables related to the adequacy of infrastructure as follows: (1) training, (2) space/equipment, (3) video software, (4) clinical support, and (5) technical support. These three factors account for 66% of the variance in provider well-being and 59% of the variance in provider desire for future telehealth.

Each factor has high internal consistency with composite reliability of 0.91, 0.89, and 0.82, exceeding the recommended level of 0.70,[29] and average variance extracted (AVE) of 0.78, 0.62, and 0.61, exceeding the recommended level of 0.50,[30] respectively, for patient effectiveness, resources, and provider satisfaction. The absence of variable cross-loadings and moderate between factor correlations of 0.66 (“Provider Satisfaction” and “Patient Effectiveness”), 0.41 (“Patient Effectiveness” and “Resources”), and 0.37 (“Provider Satisfaction” and “Resources”) suggest each factor is distinct enough to be a separate construct.

Structural equation modeling revealed significant relationships between “Resources” and “Patient Effectiveness” and between “Provider Satisfaction” and “Patient Effectiveness.” “Resources” are related to “Provider Satisfaction” in a statistically significant way, but the direct relationship is much weaker than the indirect one through “Patient Effectiveness” (i.e., “Resources” to “Patient Effectiveness” then to “Provider Satisfaction”). [Figs. 3] and [4] show the structural model results with estimated path coefficients and residual variance.

Fig. 3 Relationships between provider satisfaction, provider burnout, telehealth patient effectiveness and resources.

Fig. 4 Telehealth provider satisfaction model.

Path analysis revealed an indirect relationship between “Resources” and “Provider Desire for Future Telehealth.” There was a statistically significant direct relationship between “Resources” and “Provider Burnout.” There was not a statistically significant relationship between perceived “Patient Effectiveness” and “Provider Burnout.”

Discussion

The Pediatric Provider Telehealth Work and Well-being Survey offers one of the first large scale assessments to focus on the provider telehealth experience and associations between telehealth use and provider well-being. Our findings identify potential benefits of telehealth for provider well-being across roles and specialties. Structural equation modeling demonstrates provider perception of patient telehealth effectiveness that is significantly associated with a provider's telehealth-attributable well-being. Consistent with “Patient Well-Being” and the “Clinician-Patient Relationship” as central constructs in the NAM model, our findings indicate telehealth's capability to engage patients to meet their needs and enable delivery of high-quality care contributes importantly to provider satisfaction with telehealth. Path analysis reveals close ties between the patient-provider relationship and provider professional satisfaction. Notably, the importance of telehealth resources to provider well-being seems to be largely indirect, mediated by the relationship between resources, and perceived telehealth patient effectiveness.

Our study results confirm worrisome widespread stress among pediatric providers, as well as prevalent indications of early burnout. Though pre-COVID-19 burnout estimates are not available for our sample population, the overall rate of burnout (33%) is compatible with existing literature.[31] The single-item burnout measure used in this study, incorporated based on its validation as a stand-alone measure, inclusion in several major studies, and minimal respondent burden, is useful for clinical assessment but tends to under-identify burnout.[32]

Providers reporting burnout in this study were not ascribing such symptoms to telehealth, whereas they were specifically estimating improvement in well-being attributable to the rapid shift to telehealth. In our analysis, provider burnout was not directly related to telehealth effectiveness, but was instead directly related to telehealth resources. We interpret this to reflect burnout arising from perceived demands that chronically outweigh perceived resources.[33] The high-residual variance in burnout (0.92) in our structural model indicates that there are other significant burnout-impacting factors that this survey did not capture. Our model also underscores that clinician well-being is not merely the absence of burnout, and each is sensitive to specific determinants.

Health care organizations were obliged to accelerate implementations of telehealth solutions during the public health crisis, without the planning and stakeholder involvement that typically characterize such operational endeavors. Our findings highlight opportunities with actionable potential to sustain use of telehealth and advance provider well-being.

First, health care systems are likely to benefit from investing in robust telehealth training for provider and patient users to promote skills acquisition and enhancement. The National Consortium of Telehealth Resource Centers and groups including the American Board of Pediatrics and the American Association of Medical Colleges are developing telehealth competency standards and curricula for health professionals and students.[34] Further research is needed to evaluate telehealth content and instructional modes such as e-learning and simulation in meeting the needs of adult learners. Patient-directed education efforts are also needed to prepare patients and families for digital literacy navigation. As virtual service offerings evolve, telehealth usability testing with at-risk patient populations should be considered.

Second, our data suggest both proactive and on-demand technical supports are important to successful telehealth programs. Connectivity and audio/visual performance issues are prevalent, pernicious challenges to telehealth success. Designing clinical processes to anticipate and troubleshoot technical issues and making staff available to resolve problems has potential to minimize disruption to care. Healthcare organizations must assess and respond to broadband access and other prerequisite challenges in the communities they serve to support initial and ongoing telehealth use.

Third, providers indicated clinical workflows require reengineering for telehealth-only, as well as hybrid telehealth/in-person, care models. While telehealth is at a disadvantage in detecting many physical examination findings and performing procedural interventions, management of diverse acute and chronic conditions can leverage a combination of in-person and virtual services, and providers are interested in accommodating both. In narrative comments, providers reported challenges with follow-up of tasks emerging from video visits, likely reflecting a need to optimize immature teamwork processes specific to telehealth. Providers also described challenges maintaining clinic momentum when telehealth and in-person visits are intermixed, noting different pacing of appointment types complicates transitions between patients. Attention should be given to decision tool development to guide modality choice, implementation of appropriate provider schedules and support staffing ratios to permit smooth throughput, and clear lines of health team communication and coordination during and after video visits. Providers reported prolonged performance of telehealth led to eye strain and screen fatigue. Virtual visit “bookending,” scheduling telehealth appointments at the start or end of the day, might enable short blocks of consecutive video visits to achieve efficiency and productivity, as well as work–life flexibility.

Limitations

Our study findings should be interpreted considering key limitations. Statistical modeling was employed to explore plausible pathway associations, but experimental studies are required to establish causality among constructs. Certain survey variables were excluded from structural modeling because dichotomization produced low-frequency items that the analysis could not support. This study was conducted during the height of the COVID-19 pandemic. Telehealth was at times the only available modality for nonemergent care, possibly inflating ratings for service delivery, and patient engagement (compared with an implicit alternative of no/foregone care). Our sample did not permit meaningful subanalysis by extent of telehealth use. Self-reported outcome measures create potential for social desirability bias and recall error. Additionally, voluntary samples introduce self-selection bias. The majority of respondents practiced in multiple environments; however, we did not specifically assess whether telehealth was available or leveraged in all practice settings. Our results are most applicable to ambulatory pediatric providers and may not be generalizable to nonpediatric providers or other clinical settings. Finally, the pandemic disrupted providers' personal, as well as professional lives, although the former was not specifically investigated in our analysis.

Conclusion

System-based commitments to provider's well-being create resilient organizations. Evaluations of the recent telehealth surge and its lessons are beginning to accumulate and should inform data-driven and evidence-based best practices improvement efforts. This study offers evidence that telehealth can promote rather than detract from professional well-being for providers by enabling provision of high quality, patient-centric care and offering added flexibility and work–life balance to providers. We found widespread provider's desire to continue telehealth related to a positive perception of how effectively the medium met their patient's needs which, in turn, was influenced by how well an institution provided necessary technical and clinical resources for providers to do their jobs well. Optimizing telehealth structures and workflows to improve reliability, efficiency, and clinical excellence will benefit individuals and health care institutions.

Clinical Relevance Statement

Providers who transitioned to telehealth during the novel coronavirus disease 2019 (COVID-19) pandemic have a desire to continue telehealth. Associated factors for provider well-being in a telehealth care delivery model include provider-perceived telehealth effectiveness and adequacy of telehealth resources. Health systems can use these findings to optimize telehealth structures and workflow while planning for a postpandemic care delivery model.

Multiple Choice Questions

1. According to the structural equation model, what factor is associated with telehealth-attributable provider wellbeing?

   • Perceived effectiveness of telehealth

   • Working closely with a trainee

   • Working at an academic medical center

   • Being a subspecialist

   Correct Answer: The correct answer is option a. The factors associated with telehealth-attributed provider well-being include provider-perceived effectiveness of telehealth and adequacy of telehealth resources.

2. The likelihood of improved telehealth-associated well-being was

   • Highest among providers who also reported one or more symptoms of burnout

   • Not statistically different between providers who endorsed and denied professional stress

   • Lower than worsened telehealth-associated well-being

   • Lower among providers of advanced career stage

   Correct Answer: The correct answer is option b. Absent symptoms of burnout, there was no statistically significant difference in likelihood of improved telehealth-associated well-being between providers who denied and those who endorsed professional stress. Providers with any degree of burnout were significantly less likely to report improved telehealth-attributed well-being. Career stage was not associated with telehealth-associated well-being.

• References

• 1 Mann DM, Chen J, Chunara R, Testa PA, Nov O. COVID-19 transforms health care through telemedicine: evidence from the field. J Am Med Inform Assoc 2020; 27 (07) 1132-1135
• 2 Wosik J, Fudim M, Cameron B. et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc 2020; 27 (06) 957-962
• 3 Eschenroeder HC, Manzione LC, Adler-Milstein J. et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc 2021; 28 (05) 960-966
• 4 Friedberg MW, Chen PG, Van Busum KR. et al. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Rand Health Q 2014; 3 (04) 1
• 5 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
• 6 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
• 7 Wu DTY, Xu C, Kim A, Bindhu S, Mah KE, Eckman MH. A scoping review of health information technology in clinician burnout. Appl Clin Inform 2021; 12 (03) 597-620
• 8 Hartzband P, Groopman J. Physician burnout, interrupted. N Engl J Med 2020; 382 (26) 2485-2487
• 9 Mehta S, Machado F, Kwizera A. et al. COVID-19: a heavy toll on health-care workers. Lancet Respir Med 2021; 9 (03) 226-228
• 10 Reliford A, Adebanjo B. Use of telepsychiatry in pediatric emergency room to decrease length of stay for psychiatric patients, improve resident on-call burden, and reduce factors related to physician burnout. Telemed J E Health 2019; 25 (09) 828-832
• 11 Chu-Weininger MY, Wueste L, Lucke JF, Weavind L, Mazabob J, Thomas EJ. The impact of a tele-ICU on provider attitudes about teamwork and safety climate. Qual Saf Health Care 2010; 19 (06) e39
• 12 Kuo KM, Talley PC, Lee CM, Yen YC. The influence of telemedicine experience on physicians' perceptions regarding adoption. Telemed J E Health 2015; 21 (05) 388-394
• 13 Sauers-Ford HS, Hamline MY, Gosdin MM. et al. Acceptability, usability, and effectiveness: a qualitative study evaluating a pediatric telemedicine program. Acad Emerg Med 2019; 26 (09) 1022-1033
• 14 Brigham T, Barden C, Dopp AL. et al. A journey to construct an all-encompassing conceptual model of factors affecting clinician well-being and resilience. Accessed January 11, 2022: https://nam.edu/wp-content/uploads/2018/01/Journey-to-Construct-Conceptual-Model.pdf
• 15 Doarn CR, Pruitt S, Jacobs J. et al. Federal efforts to define and advance telehealth–a work in progress. Telemed J E Health 2014; 20 (05) 409-418
• 16 Weaver MS, Lukowski J, Wichman B, Navaneethan H, Fisher AL, Neumann ML. Human connection and technology connectivity: a systematic review of available telehealth survey instruments. J Pain Symptom Manage 2021; 61 (05) 1042-1051.e2
• 17 deMayo R, Heggland A, Grindle C. et al. Pediatric telemedicine work and well-being survey. Accessed May 31, 2021: https://is.gd/peditelemed
• 18 Arch collaborative example survey. Available at https://klasresearch.com/images/pages/usability-studies/base-emr-experience-survey.pdf
• 19 Sisk B, Alexander J, Bodnar C. et al. Pediatrician attitudes toward and experiences with telehealth use: results from a national survey. Acad Pediatr 2020; 20 (05) 628-635
• 20 Yip MP, Chang AM, Chan J, MacKenzie AE. Development of the Telemedicine Satisfaction Questionnaire to evaluate patient satisfaction with telemedicine: a preliminary study. J Telemed Telecare 2003; 9 (01) 46-50
• 21 Parmanto B, Lewis Jr. AN, Graham KM, Bertolet MH. Development of the Telehealth Usability Questionnaire (TUQ). Int J Telerehabil 2016; 8 (01) 3-10
• 22 Bakken S, Grullon-Figueroa L, Izquierdo R. et al; IDEATel Consortium. Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. J Am Med Inform Assoc 2006; 13 (06) 660-667
• 23 Dolan ED, Mohr D, Lempa M. et al. Using a single item to measure burnout in primary care staff: a psychometric evaluation. J Gen Intern Med 2015; 30 (05) 582-587
• 24 random.polychor.pa: A Parallel Analysis with Polychoric Correlation Matrices. Accessed May 31, 2021: https://CRAN.Rproject.org/package=polycor
• 25 Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research. R package version 2.1.3. Northwestern University:. 2021 Accessed May 31, 2021 at: https://CRAN.R-project.org/package=psych
• 26 Rosseel Y. lavaan: An R Package for Structural Equation Modeling. J Stat Softw 2012; 48 (02) 1-36
• 27 West CP, Dyrbye LN, Sloan JA, Shanafelt TD. Single item measures of emotional exhaustion and depersonalization are useful for assessing burnout in medical professionals. J Gen Intern Med 2009; 24 (12) 1318-1321
• 28 Brady KJS, Ni P, Carlasare L. et al. Establishing crosswalks between common measures of burnout in US physicians. J Gen Intern Med 2021;
• 29 Hair Jr. J, Sarstedt M, Hopkins LG, Kuppelwieser V. Partial least squares structural equation modeling (PLS-SEM). Eur Bus Rev 2014; 26 (02) 106-121
• 30 Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 1981; 18 (01) 39-50
• 31 Rotenstein LS, Torre M, Ramos MA. et al. Prevalence of burnout among physicians: a systematic review. JAMA 2018; 320 (11) 1131-1150
• 32 West CP, Dyrbye LN, Satele DV, Sloan JA, Shanafelt TD. Concurrent validity of single-item measures of emotional exhaustion and depersonalization in burnout assessment. J Gen Intern Med 2012; 27 (11) 1445-1452
• 33 Demerouti E, Bakker AB, Nachreiner F, Schaufeli WB. The job demands-resources model of burnout. J Appl Psychol 2001; 86 (03) 499-512
• 34 AAMC. Telehealth competencies across the learning continuum. Accessed January 11, 2022 at: https://store.aamc.org/downloadable/download/sample/sample_id/412/