Directly Integrating Health Information Exchange (HIE) Data with the Electronic Health Record Increases HIE Use by Emergency Department Clinicians

Abstract

Objectives This article (1) develops a Fast Healthcare Interoperability Resources app, Health Dart, that integrates information from Indiana's community health information exchange (HIE), the Indiana Network for Patient Care (INPC), directly with Cerner, an electronic health record (EHR), and (2) evaluates the effect of Health Dart's implementation on HIE use.

Methods Health Dart was implemented in 14 Indiana University Health emergency departments (EDs) using a stepped-wedge study design. We analyzed rates of INPC use in 286,175 ED encounters between October 1, 2019 and December 31, 2020. Logistic regression was used to model the probability of INPC use given the implementation context, such as user interface (UI) enhancements and the coronavirus disease 2019 pandemic.

Results INPC use increased by 131% across all encounters (from 3.6 to 8.3%; p < 0.001) after Health Dart implementation. INPC use increased by 144% (from 3.6 to 8.8%; p < 0.001) more than 2 months postimplementation. After UI enhancements, postimplementation INPC use increased by 123% (from 3.5 to 7.8%) compared to 181% (from 3.6 to 10.1%; p < 0.001) in postimplementation encounters that occurred before UI enhancements. During the pandemic, postimplementation INPC use increased by 135% (from 3.4 to 8.0%; p < 0.001) compared to 178% (from 3.6 to 10%; p < 0.001) in postimplementation encounters that occurred before the pandemic. Statistical significance was determined using 95% confidence intervals (α = 0.05).

Conclusion Direct integration of HIE information into an EHR substantially increased frequency of HIE use, but the effect was weakened by the UI enhancements and pandemic. HIE information integrated into EHRs in the form of problem-oriented dashboards can potentially make information retrieval more efficient and effective for clinicians.

Keywords

emergency departments - health information exchange - Fast Healthcare Interoperability Resources - FHIR - electronic health record - COVID-19 pandemic

Background and Significance

Providing clinicians with the information they need when and in the format they need it is a significant research and operational challenge in clinical informatics.[1] [2] [3] [4] Information needed to care for patients is typically still fragmented across too many different systems and not easy to collate, organize, and review.[1] [5] [6] This problem is aggravated in the context of increasing interoperability. Not only are clinicians tasked with thoroughly reviewing patient cases in their own electronic health record (EHR), but they are also expected to do the same for patient records generated outside of their organization available through health information exchange (HIE).

HIE provides many benefits for health care processes and outcomes, which is why it is strongly supported by the Office of the National Coordinator and multiple other stakeholders.[7] [8] [9] [10] [11] [12] [13] [14] Among these benefits are fewer duplicated procedures, reduced imaging, lower costs, and improved patient safety.[7] [15] Despite these benefits, the majority of hospitals do not engage in meaningful integration of shared health data in the EHR beyond the continuity of care record.[16] [17]

Clinicians ideally need all relevant information about a specific patient in one place.[18] Missing (or inaccessible) information can have detrimental consequences for care.[19] Community HIEs typically provide broad, comprehensive coverage of patient information from health care organizations in a region, regardless of which EHR they use.[20] Vendor-mediated HIEs, on the other hand, such as Care Everywhere in Epic and CommonWell in Cerner, and collaboratives such as Carequality are often more constrained in their information coverage to within their customer base or across shared products. These constraints sometimes make it difficult or impossible for clinicians to access all relevant information about a patient.[20]

Clinicians typically access information in the HIE through a separate application or portal because most HIEs are not directly integrated with EHRs. However, the resulting multiple logins, workflow interruptions, and poor location and presentation of the HIE user interface (UI) often impede effective and efficient information retrieval, resulting in a low level of use, especially in emergency medicine.[21] [22] [23] [24] [25] [26] Vendor-mediated HIEs often preferentially integrate external information in the EHR only when it comes from their customers, leaving information from other EHRs more difficult to access.[20] Early evidence shows that integrating HIE information directly into the EHR through an easily accessible UI can substantially increase the use of information from the HIE.[25] [26]

Retrieving patient information through most HIE portals is subject to an additional limitation: clinicians must typically browse through information organized by type, such as medications, labs, orders or physician notes, and time, because like EHRs, most HIEs do not offer problem-oriented views.[27] These limitations have two major consequences. First, the lack of widescale interoperability and barriers to information access impose an exhausting litany of clerical tasks on clinicians which contributes to burnout and waste.[28] [29] Second, clinicians routinely forgo searching for and retrieving additional clinical data about patients,[30] which contributes to waste and adverse patient outcomes. Problem-oriented views have been shown to improve data retrieval workflows, allowing providers to complete EHR tasks more efficiently, with fewer errors and cognitive task load, and greater user satisfaction.[31] [32] [33] [34] The importance of these benefits was highlighted during the coronavirus 2019 (COVID-19) pandemic, when emergency departments (EDs) became burdened with an overwhelming number of patients. Given the severity of COVID-19 and emergency medicine as a primary point of care, the pandemic provides an opportunity to study how HIE use may have changed during this period of heightened stress in EDs.

In this study's context of Indiana's HIE, the Indiana Network for Patient Care (INPC), clinicians' traditional access to HIE is through CareWeb, a single sign-on Web-based viewer with a search function that displays the patient's community health record. After invoking CareWeb from PowerChart [*], clinicians must browse or search the patient's health records to find relevant information. A major strength of the INPC, the nation's largest community HIE, is its comprehensiveness; however, the lack of a problem-oriented view and EHR integration reinforces the same limitations common to other HIEs. The Fast Healthcare Interoperability Resources (FHIR) standard, on the other hand, allowed us to retrieve information from the INPC and display it in a problem-oriented dashboard within Cerner. In general, FHIR enables a convenient and meaningful way to share, integrate, and use curated patient health information in addition to the option of taking a deeper dive into the patient health record using an HIE portal.

Objectives

The primary goal of this study, therefore, was twofold: (1) develop a FHIR app, Health Dart, that integrates information from Indiana's community HIE, the INPC, directly with an EHR in the form of a problem-oriented dashboard; and (2) determine how the rollout of Health Dart at 14 Indiana University Health (IU Health) EDs affected overall INPC use among clinicians using a stepped-wedge trial design, allowing us to causally relate Health Dart with changes in overall INPC use regardless of access mode (i.e., FHIR app or Web-based viewer).[35] [36]

Methods

We developed, deployed, and evaluated the effect of Health Dart using a stepped-wedge design. In this design, Health Dart was rolled out in a staggered fashion across the ED sites (i.e., in waves of three to four facilities at a time). This design allowed for each study site to have both a pre- and postintervention observation period while allowing the later wave sites to serve as concurrent controls. The stepped-wedge approach also facilitated the rollout by introducing Health Dart at a manageable number of sites at the same time.[35] [36]

Health Dart Development

We developed the Health Dart application (app) ([Fig. 1]) based on an earlier version focused on chest pain.[25] The goal of developing Health Dart was to automatically retrieve information from the INPC most relevant to selected chief complaints and integrate it directly into PowerChart in the form of a chief complaint-focused dashboard. We were unable to comingle information from the INPC in Cerner similarly to Epic and Care Everywhere because of technical constraints. The [Supplementary Appendix] (available in the online version) provides a detailed description of the development and software/system architecture of Health Dart ([Fig. 2]).

Fig. 1 The Health Dart app within PowerChart, the Cerner EHR, showing HIE information relevant to chest pain. EHR, electronic health record; HIE, health information exchange.

Fig. 2 Software architecture and application workflow of Health Dart. The workflow consists of the following steps: Cerner PowerChart loads Health Dart into a dedicated frame, and passes the provider credentials, along with the patient identifier, to the app (1). Health Dart posts a request (using the HTML POST method) for data to the Back-end for Front-end Service (2). The Back-end for Front-end Service forwards the user credentials and context to the Authentication Processor (3). The Processor checks the request against various business rules, such as whether the user is authorized to use Health Dart and access data about the specific patient. The Processor logs the login request (4) and verifies the credentials (5). The result of the credential verification is returned to the Back-end for Front-end Service (6) that then communicates with the API Server to retrieve data (7) if the user is authorized to use Health Dart and access data. (If that is not the case, the Back-end for Front-end Service returns an error message to Health Dart [not shown].) The API Server comprises a number of APIs, including a commercial FHIR API (Mulesoft, San Francisco, California, United States). The FHIR Server connects to the Indiana Health Information Exchange Database to query the requested resources (8), which provides them to the API Server (9). The API Server communicates them to the Back-end for Front-end Service (10). After performing any transformations needed, the requested content is provided to the app (11). The response body is used by the app to place content in the PowerChart frame (12).

Pilot Implementation, Study Design, and Setting

Health Dart was implemented at a central pilot site, IU Health Methodist ED, in January 2018 before its launch across the remaining 14 IU Health EDs with 166 clinicians using a stepped-wedge, cluster nonrandomized controlled study design beginning in December 2019. As an alternative to randomized controlled trials, which are often not practical for site-based studies of health care delivery interventions, this scientifically validated, pragmatic study design enables causal inference from the multi-site intervention rather than merely establishing an association.[37] [38] [39] Health Dart was available at the bottom of the navigation pane in Cerner PowerChart across all sites in April 2018 due to technical limitations prohibiting deployment to individual sites. The rollout across the remaining 14 ED sites was delayed to prepare for the stepped-wedge trial and acquire the necessary funding.

Formal rollout, which included advertisement via email and in-person or virtual training on where to find and how to use the app, began in December 2019 with the first wave, or cluster, of ED sites. (As evidence of the lack of awareness by clinicians of Health Dart, the app was used only 304 times during 286,175 encounters [0.1%] prior to the app rollout in each wave.) Rivera, assisted by Hosler, led the formal rollout in partnership with the IU Health Chief Medical Information Officer, Schaffer, and ED site directors. Rollout occurred in four waves, comprising three to four ED sites located in the same geographic region. This sampling technique minimized contamination across waves due to clinicians who were employed in multiple study sites. For each wave, we assessed HIE use before the formal rollout, and during the 0 to 2, as well as >2 months after the rollout. The stepped-wedge cluster design allowed for each wave to contribute both control and exposed observations. Prior to rollout, each wave provided control observations sequentially followed by exposed observations by wave over time (in steps), until all waves crossed over from control to exposed. Finally, each wave provided an additional time period of observations when all waves were exposed. The robustness of this pragmatic study design enables causal inference about the effects of Health Dart on INPC use. [Fig. 3] depicts the rollout sequence across the four waves.

Fig. 3 Monthly rates of HIE use in each Health Dart rollout wave. HIE use is defined as accessing the HIE either through Health Dart or the Web-based application during a patient encounter. Health Dart was implemented in four Wave 1 sites on December 13, 2019; three Wave 2 sites on February 7, 2020; three Wave 3 sites on June 8, 2020; and four Wave 4 sites on August 3, 2020. The rollout study was temporarily paused between Wave 2 and Wave 3 due to the COVID-19 pandemic. A total of 110 encounters had missing data (0.04%). Due to technical limitations, the Health Dart application was available in Cerner Powerchart across all sites in April 2018; therefore, we could not prevent use of the application prior to the study period for each wave. HIE, health information exchange; UI, user interface.

Conceptual Models

We used the Unified Theory of Acceptance and Use of Technology (UTAUT) and Consolidated Framework for Implementation Research (CFIR) as our conceptual models. Each of these models acknowledges the intervention, user perceptions and attitudes of the intervention, social influence, implementation processes and setting as predictors of user behaviors, adoption, and sustainable use of HIE.[40] [41] [42] Together, these factors constitute the rollout context (the setting and circumstances of the implementation).[40] [41] [42] In the analysis of the effect of the rollout context of Health Dart on INPC use, we focused on examining the effect of changes made to the app, including enhancements to the UI and repositioning of the app in the toolbar, and the COVID-19 pandemic. Although user perceptions and attitudes were collected via survey, and clinical outcomes were extracted from EHR data, they are not presented here because they are not a focus of this study.

Dependent and Independent Variables, and Measures

The dependent variable in this study was HIE use through either Health Dart or the INPC. We did not break out Health Dart or INPC use separately because our clinical stakeholders were primarily interested in facilitating the use of information in the HIE, regardless of access mechanism. The primary independent variable was the integration of HIE information in the EHR in the form of a dashboard through Health Dart (binary). Secondary independent variables, collectively termed “rollout context,” are described above.

The COVID-19 pandemic was an unplanned but important addition to our study variables. The National Syndromic Surveillance Program reported a 42% decline in ED visits between March 29 and April 25, 2020, compared to the same time period in 2019.[35] During this time, many common ED chief complaints were displaced by infectious diseases and respiratory conditions compared to the prior year,[35] [36] making significant changes in HIE use patterns likely.

We anticipated that Health Dart rollout would cause an incremental increase in overall HIE use, followed by a leveling off over time. We predicted a similar use pattern following the UI enhancements and the repositioning of the app in the toolbar. Because there was a reduction in opportunities to use Health Dart for its intended purpose due to COVID-19, we expected an initial decrease in HIE use shortly after the start of the pandemic with a gradual increase and leveling off over time.

We evaluated INPC use for the 286,175 ED encounters that occurred between October 1, 2019 and December 31, 2020, at all sites collectively. User log data and encounter details were extracted from the INPC and IU Health Enterprise Data Warehouse by the Regenstrief Institute Data Core. Encounters were categorized by rollout wave (Wave 1–4), INPC use (yes or no), and INPC access method (CareWeb or Health Dart). Encounters were also categorized as occurring pre- or post-rollout, before or after UI enhancements were made to the app (May 11, 2020), before or after the app was moved to a more prominent position on the toolbar (October 14, 2020), and before or during the COVID-19 pandemic as defined by the date of the first reported COVID-19 case in Indiana (March 6, 2020). To understand the temporal effects of app rollout, encounters were also grouped by three different time periods relative to rollout: pre-rollout, between 0 and 2 months post-rollout, and >2 months post-rollout. The outcome was a binary variable representing whether the INPC (either through Health Dart or CareWeb) was used in an encounter.

Statistical Analysis

First, a descriptive analysis provided rates of INPC use for each ED site and rollout wave over the study period (October 1, 2019 to December 31, 2020). For each site, we also determined the proportion of INPC use that occurred through Health Dart versus CareWeb.

Second, we determined the associations between INPC use and variables describing the rollout context. Using a chi-square test, we compared INPC use rates between pre- and post-Health Dart rollout groups for all encounters and stratified them by the rollout wave, UI enhancements, toolbar repositioning, and COVID-19 pandemic. Chi-square and Cochran–Armitage Trend tests were used to compare and assess changes in INPC use rates across the three time periods (pre, 0–2 months, and >2 months) for all encounters, stratified by rollout wave, UI enhancements, and COVID-19 pandemic. A chi-square test was used to assess whether increases in INPC use were associated with the repositioning of the app on the toolbar. Because this change was made after Health Dart was implemented in all four waves, we were not able to compare INPC use rates among the three time periods, and thus could not separate the effect of the app rollout from the toolbar repositioning.

Third, logistic regression modeling was used to estimate the effect of Health Dart rollout on the probability of INPC use while adjusting for rollout wave, UI enhancements, and pandemic status. Rollout wave was included as a covariate to reflect the temporal trend under the stepped-wedge design because Health Dart was rolled out at different time periods for different waves. We used four models: Model A: the binary Health Dart rollout variable and rollout wave; Model B: variables in Model A, the binary variables of the UI enhancement, and pandemic status, and their interactions with the Health Dart rollout variable; Model C: the three-level categorical variable of different Health Dart rollout time periods and rollout waves; and Model D: variables in Model C, the binary variables of the UI enhancement and pandemic status, and their interactions with the three-level categorical variable of different Health Dart rollout time periods. In all models, a site-level random intercept was included to account for clustering effects within the same sites. Treating the ED site as a random effect allowed us to incorporate the variability in the site effect that was due to selecting a limited set of ED sites where Health Dart might have been tested. For example, Health Dart may be rolled out to additional health system EDs outside of IU Health, but for this study we selected a sample of ED sites from all potential ED sites in Indiana or the United States.

A power analysis was conducted based on the stepped-wedge cluster nonrandomized design and pilot study results to test the hypothesis that app rollout increased INPC use.[25] [37] [38] [39] A study site having at least 39 daily encounters per 2-month rollout wave (a total of 2,340 encounters) would result in the detection of a change in the rate of HIE access and the conclusion that Health Dart led to greater INPC usage under a stringent test with a significance level of α ≤ 0.01 powered at 0.9. Given the large encounter volume at IU Health EDs (286,175 ED encounters) during the study period, there was adequate power to detect a statistically significant difference in INPC use due to the app rollout. Analyses were completed using SAS 9.4 (SAS Institute Inc., Cary, North Carolina, United States). Statistical significance was determined using 95% confidence intervals (alpha level = 0.05).

Results

Health Dart Development

Health Dart is a FHIR-based “app” that integrates information from the INPC with IU Health's ED EHR system, Cerner PowerChart. Clinicians' traditional way of accessing the INPC through CareWeb,[43] a Web-based viewer, remains available if clinicians need to review the full patient record in the CareWeb application. This dashboard provides 8 to 12 data elements highly relevant to seven chief complaints (chest pain, abdominal pain, weakness/dizziness/headache, back/flank pain, pregnancy, arrhythmia, and dyspnea) ([Fig. 1]). This integration significantly reduced the time and number of clicks needed to access HIE information directly in the EHR.[25]

INPC Use by Rollout Wave

The average rate of INPC use at the 14 sites over the study period (October 1, 2019 to December 31, 2020) ranged from 1.2% in Wave 3 to 18.1% in Wave 1. Rates were similar among EDs in the same rollout wave except for IU Health Morgan in Wave 4, which used the INPC at a higher rate than the other Wave 4 sites particularly during March to April of 2020 and prior to their Health Dart roll-out. In all waves, INPC use initially increased after rollout and continued to increase after UI enhancements and toolbar repositioning. The odds of INPC use increased 10% after the toolbar changes (p < 0.001). In Wave 1, there was a decline in use of the INPC in March 2020 coinciding with the start of the COVID-19 pandemic with a gradual return to prepandemic rates over time. The effect of the pandemic was not as dramatic at the other sites ([Fig. 3]). Across all four rollout waves, there was a pattern of incremental increases in INPC use in the 0 to 2 months and >2 months post-rollout time periods. At four sites (IU Health Tipton, Blackford, Jay, and Paoli), INPC use leveled off between the 0 to 2 months and >2 months post-rollout time periods ([Fig. 4]).

Fig. 4 Rate of HIE use before and after Health Dart rollout at 14 IU Health EDs in four rollout waves. HIE use combined access through the Web-based viewer and Health Dart. ED, emergency department; HIE, health information exchange; IU, Indiana University.

INPC Use before and after Health Dart Rollout

Clinicians' INPC use was 131% higher overall (3.6 vs. 8.3%; p < 0.001) in post-rollout encounters compared to pre-rollout encounters ([Table 1]). The increased likelihood of INPC use in post-rollout encounters was also observed in each rollout wave, in pre- and post-UI enhancement periods, and before and during the COVID-19 pandemic. The effect of the rollout was not as strong in encounters that occurred after UI enhancements or during the pandemic. After UI enhancements, post-rollout INPC use increased by 123% (from 3.5 to 7.8%; p < 0.001) compared to 181% (from 3.6 to 10.1%; p < 0.001) in post-rollout encounters that occurred before UI enhancements. During the pandemic, post-rollout INPC use increased by 135% (from 3.4 to 8.0%; p < 0.001) compared to 178% (from 3.6 to 10%; p < 0.001) in post-rollout encounters that occurred before the pandemic.

Table 1 Comparison of INPC use pre- and post-Health Dart rollout

ED encounters[a]	N	(%)	Pre-	Post-	Increase[b]	ꭓ 2 p-value[c]	OR[d]	[95% CI]
Total	286,175	(100.0)	3.6%	8.3%	131%	<0.001	2.44	[2.35–2.52]
Wave 1	72,639	(25.4)	9.2%	13.0%	41%	<0.001	1.48	[1.39–1.58]
Wave 2	63,667	(22.2)	4.4%	6.7%	52%	<0.001	1.58	[1.46–1.71]
Wave 3	61,715	(21.6)	1.8%	3.0%	67%	<0.001	1.73	[1.56–1.93]
Wave 4	88,154	(30.8)	3.1%	5.9%	90%	<0.001	1.95	[1.82–2.08]
Pre-UI enhancements	143,024	(50.0)	3.6%	10.1%	181%	<0.001	3.03	[2.89–3.17]
Post-UI enhancements	143,151	(50.0)	3.5%	7.8%	123%	<0.001	2.29	[2.12–2.49]
Pre-COVID-19	111,504	(39.0)	3.6%	10.0%	178%	<0.001	2.94	[2.78–3.12]
During COVID-19	174,671	(61.0)	3.4%	8.0%	135%	<0.001	2.50	[2.35–2.66]

Abbreviations: CI, confidence interval; ED, emergency department; INPC, Indiana Network for Patient Care; OR, odds ratio; UI, user interface.

^a Data are derived from the IU Health Data Warehouse covering 14 emergency departments. Total encounters were reported from October 1, 2019 to December 31, 2020. Encounters with missing data (n = 110; 0.04%) were excluded from analysis.

^b The overall percentage change may be very high even with relatively low percentage changes in the subgroups that have unbalanced numbers of encounters between pre-Health Dart and post-Health Dart rollout and varying group-specific INPC usage rates.

^c Chi-square tests compared rates of INPC use between pre- and post-Health Dart rollout.

^d Odds ratios represent the odds of INPC use post-Health Dart rollout divided by the odds of INPC use pre-Health Dart rollout.

As was observed in the descriptive analysis, INPC use rates over the 0 to 2 months post-rollout and >2 months post-rollout time periods increased across all encounters compared to pre-rollout (p < 0.001) ([Table 2]). This trend was also observed in each rollout wave, in pre- and post-UI enhancement periods, and before and during the COVID-19 pandemic. In each of the time periods, the effect of Health Dart on INPC use was not as strong in encounters that occurred after UI enhancements (p < 0.001) or in encounters that occurred during COVID-19.

Table 2 Comparison of INPC use stratified by pre-, 0–2 months post-, and more than 2 months post-Health Dart rollout time periods

			Health Dart time periods[a]
ED encounters[b]	N	(%)	Pre-	0–2 months post	>2 months post	ꭓ 2 p-value[c]	CA p-value[d]
Total	286,175	(100.0)	3.6%	6.5%	8.8%	<0.001	<0.001
Wave 1	72,639	(25.4)	9.2%	11.5%	13.4%	<0.001	<0.001
Wave 2	63,667	(22.2)	4.4%	5.6%	6.9%	<0.001	<0.001
Wave 3	61,715	(21.6)	1.8%	2.6%	3.2%	<0.001	<0.001
Wave 4	88,154	(30.8)	3.1%	5.0%	6.4%	<0.001	<0.001
Pre-UI enhancements	143,024	(50.0)	3.6%	9.0%	11.4%	<0.001	<0.001
Post-UI enhancements	143,151	(50.0)	3.5%	4.1%	8.5%	<0.001	<0.001
Pre-COVID-19	111,504	(39.0)	3.6%	9.9%	10.3%	<0.001	<0.001
During COVID-19	174,671	(61.0)	3.4%	4.3%	8.8%	<0.001	<0.001

Abbreviations: CA, Cochran–Armitage Trend test; ED, emergency department; INPC, Indiana Network for Patient Care; UI, user interface.

^a To test for temporal trends in INPC use, three time periods were used: pre-Health Dart rollout, 0–2 months post-Health Dart rollout, and >2 months post-Health Dart rollout.

^b Data were derived from the IU Health Data Warehouse covering 14 emergency departments. Total encounters were reported from October 1, 2019 to December 31, 2020. Encounters with missing data (n = 110; 0.04%) were excluded from analysis.

^c Chi-square tests compared rates of INPC use between pre- and post-Health Dart rollout.

^d Cochran–Armitage Trend tests determined whether INPC use increased over the three time periods.

Estimating the Probability of INPC Use

The odds of INPC use increased 68% post-Health Dart rollout, adjusted for the rollout wave (p < 0.001) ([Table 3]). The odds of INPC use was 47% higher at 0 to 2 months post-rollout (p < 0.001) and 76% higher at >2 months post-rollout (p < 0.001) compared to pre-Health Dart, after controlling for wave.

Table 3 Modeling the probability of INPC use

Model[a]		OR	[95% CI]	p-Value
A	Post-Health Dart rollout	1.68	[1.62–1.75]	<0.001
	Pre-Health Dart rollout	Reference
B	Post-Health Dart rollout at pre-UI enhancements	1.38	[1.29–1.47]	<0.001
	Pre-Health Dart rollout at pre-UI enhancements	Reference
	Post-Health Dart rollout at post-UI enhancements	1.47	[1.33–1.64]	<0.001
	Pre-Health Dart rollout at post-UI enhancements	Reference
	Post-Health Dart rollout at pre-COVID-19	1.46	[1.33–1.59]	<0.001
	Pre-Health Dart rollout at pre-COVID-19	Reference
	Post-Health Dart rollout during COVID-19	1.39	[1.29–1.50]	<0.001
	Pre-Health Dart rollout during COVID-19	Reference
C	0–2 months after Health Dart rollout	1.47	[1.40–1.55]	<0.001
	>2 months after Health Dart rollout	1.76	[1.69–1.83]	<0.001
	Pre-Health Dart rollout	Reference
D	0–2 months after Health Dart rollout at pre-UI enhancements	1.26	[1.15–1.39]	<0.001
	>2 months after Health Dart rollout at pre-UI enhancements	1.37	[1.26–1.48]	<0.001
	Pre-Health Dart rollout at pre-UI enhancements	Reference
	0–2 months after Health Dart rollout at post-UI enhancements	1.58	[1.37–1.81]	<0.001
	>2 months after Health Dart rollout at post-UI enhancements	1.40	[1.25–1.57]	<0.001
	Pre-Health Dart rollout at post-UI enhancements	Reference
	0–2 months after Health Dart rollout at pre-COVID-19	1.64	[1.46–1.85]	<0.001
	>2 months after Health Dart rollout at pre-COVID-19	1.31	[1.15–1.48]	<0.001
	Pre-Health Dart rollout at pre-COVID-19	Reference
	0–2 months after Health Dart rollout during COVID-19	1.21	[1.09–1.34]	0.003
	>2 months after Health Dart rollout during COVID-19	1.46	[1.35–1.58]	<0.001
	Pre-Health Dart rollout during COVID-19	Reference

Abbreviations: CI, confidence interval; ED, emergency department; INPC, Indiana Network for Patient Care; UI, user interface.

^a Model A: the binary Health Dart rollout variable and rollout wave; Model B: variables in Model A, the binary variables of the UI enhancement, and pandemic status and their interactions with the Health Dart implementation variable; Model C: the three-level categorical variable of different Health Dart implementation time periods and implementation waves; and Model D: variables in Model C, the binary variables of the UI enhancement and pandemic status, and their interactions with the three-level categorical variable of different Health Dart implementation time periods.

The odds of INPC use was 38% higher after the Health Dart rollout and before UI enhancements (p < 0.001). The effect of Health Dart strengthened after UI enhancements, with the odds of INPC use increasing to 47% (p < 0.001). There was an increasing trend of odds of INPC use over time post-Health Dart rollout both before and after UI enhancements. Before the UI enhancement, the odds of INPC use was 26% higher at 0 to 2 months post-rollout (p < 0.001) and 37% higher at >2 months post-rollout (p < 0.001) compared to pre-Health Dart. After the UI enhancement, the odds of INPC use was 58% higher at 0 to 2 months post-rollout and 40% higher at >2 months post-rollout (all p-values < 0.001).

Before the COVID-19 pandemic, the odds of INPC use was 46% higher after Health Dart rollout (p < 0.001). The effect of Health Dart slightly weakened during the COVID-19 pandemic, when the odds of INPC use was 39% higher after Health Dart rollout (p < 0.001). Before the pandemic, the odds of INPC use was 64% higher at 0 to 2 months post-rollout and 31% higher at >2 months post-rollout compared to pre-rollout (all p-values < 0.001). On the other hand, during the pandemic, there was a clear trend of increasing Health Dart effect over time, with the odds of INPC use 21% higher at 0 to 2 months post-rollout (p = 0.003) and 46% higher at >2 months post-rollout (p < 0.001).

Discussion

This study had a two-part goal: (1) develop Health Dart, a FHIR app that integrates information from the INPC directly with Cerner in the form of a chief complaint-oriented dashboard; (2) determine how the rollout of Health Dart affected overall INPC use among clinicians at 14 IU Health EDs using a stepped-wedge trial design. The development of Health Dart addressed two current, major limitations of HIE implementation and use. First, it integrated relevant information from the HIE directly into the EHR as opposed to forcing clinicians to access a separate HIE system. Second, it presented this curated information in the form of a dashboard focused on a chief complaint, obviating the need for the clinician to manually collate this information.

The large increase in INPC use at IU Health resulting from the rollout of Health Dart is remarkable considering that EDs across multiple Indiana health systems reported only a 9% increase in HIE use following the passage of the 2009 HITECH Act (between 2011 and 2017) and rollout of single sign-on to the INPC starting in 2014.[44] The same study also reported HIE use increased by 29 and 3.5% in inpatient and outpatient settings, respectively, indicating that care environments may differ in terms of HIE use.[44] Our results are consistent with previous findings in the ED setting that aggregation of data from disparate sources into a single view increased the frequency with which ED clinicians accessed HIE by 91.7%.[26] Separate rates of INPC access via Health Dart versus CareWeb are not presented here because the study focus was overall HIE use, regardless of access mode (i.e., FHIR app or Web-based viewer).

Contrary to our prediction, the effect of the app's rollout on INPC use was not as strong in encounters that occurred after UI enhancements. The UI enhancements appeared to have had a temporal effect on INPC use, with an initial increase in INPC use in the short term (<2 months) followed by a return to pre-UI enhancement INPC use rates in the medium term (>2 months). The gradual effect of Health Dart rollout and UI enhancements on INPC use is supported by the Innovation Diffusion Theory, in which the use of technology increases slowly at first and then rapidly before leveling off.[45] The individual, social, and organizational factors outlined in the UTAUT model also can potentially explain why INPC use rates and methods of use differed by ED and by rollout wave. That is, clinicians in each of the EDs have different expectations (e.g., leaders, peers, and organizational and technical infrastructures) that influence their intention and behavior to use Health Dart.[40] Variations in rates of INPC use across sites and rollout waves is also consistent with EHR implementation evaluations which note how variability in rollout processes, such as physician training and timing of software updates, influence clinicians' perceptions about the usability of a newly implemented technology.[46] In our study, which focused on rollout context guided by the CFIR model, the rollout factors (i.e., UI enhancements and COVID-19 pandemic) weakened the effect of Health Dart in increasing INPC use.

Moving the Health Dart app toward the top of the Cerner PowerChart toolbar was associated with an increased likelihood of INPC use among ED clinicians. Relocation of the app to a more prominent location presumably increased its usability by decreasing the time required for users to find and use it. This finding is consistent with Fitts's law and Jakob Nielsen's usability heuristics for UI design.[47] [48] Fitts's law describes the amount of time it takes a user to complete an action as a function of the distance and accuracy of the movement, while Nielsen's usability heuristics suggest that visibility and prioritization of relevant content increase usability.[47] [48] Because the toolbar relocation occurred toward the end of the study, we were not able to separate the effects of the Health Dart rollout on INPC use from the effects of the toolbar changes. To reduce confounding of our results, we excluded encounters that occurred prior to Health Dart rollout when evaluating the effect of toolbar relocation on INPC use.

Congruent with our prediction, the effect of the rollout on INPC use was not as strong in encounters as that occurred during as opposed to prior to the COVID-19 pandemic. The pandemic had a temporal effect on INPC use, with a decline in INPC use in the short term (<2 months) followed by a return to pre-COVID-19 INPC use rates in the medium term (>2 months). The results of the COVID-19 modeling should be interpreted with caution as the lower volume of patients and higher rate of infectious diseases and respiratory conditions in encounters during the pandemic increase the potential for confounding.

This study extends our prior work[25] [44] through a larger sample size, increased statistical power, and the ability to limit the effect of confounding using the stepped-wedge trial design. Although the data were derived from only one statewide hospital system in the Midwest, this novel method can potentially be scaled to additional EDs and HIEs to improve generalizability of results for different populations, hospital systems, and EHR vendor platforms. The comprehensive and diverse data sources of the INPC also serve to improve the generalizability of our results.

A limitation of this study, similar to other evaluations of health care technology adoption and use, was not measuring the variability among individual clinicians when assessing the causal effect of the Health Dart rollout on INPC use.[7] [8] [19] [20] Models controlled for ED sites to account for differences between individual clinicians, and we assumed that clinicians working at the same ED had similar organizational and social influences. In current research, we plan to analyze the probability of INPC use given clinician, patient, and encounter characteristics; the utility of INPC access given various encounter, patient and clinician characteristics; and how clinicians decide to access Health Dart, CareWeb, or both. A more granular level of analysis may elucidate anomalies, such as IU Health Morgan's higher rates of INPC use compared to other Wave 4 sites. Additionally, because Health Dart was designed for seven chief complaints, we will examine whether there is a relationship between chief complaint and use of INPC. The user log data indicate if a user accessed INPC data, but to date we are unable to account for what information the user accessed and whether it was relevant to clinical decision-making, care provided, or clinical outcomes. Knowing what the user accessed could inform whether the actual rates of INPC use reflect the opportunities for appropriate use. Currently, how to determine optimal levels of HIE use in the ED remains unknown.

Conclusion

In summary, Health Dart increased HIE use among ED clinicians in a Midwestern hospital system. Contextual factors including UI enhancements, repositioning the app on the EHR toolbar, and the COVID-19 pandemic influenced the effect of Health Dart on HIE use.

Clinical Relevance Statement

Our results indicate that directly integrating HIE information into the EHR in a problem-oriented format can increase HIE use in the ED. In addition, they provide evidence of the influence of rollout context in the adoption and use of HIE. The results underscore the importance of considering contextual influences such as culture, policy, and setting when evaluating the rollout of a novel technology in the ED.

Conflict of Interest

None declared.

Acknowledgements

The authors acknowledge Drs. Richard Holden, Julia Adler-Milstein, Saurabh Rahurkar, and Hana Chung for their contributions to this research. Thank you to Sarah Zappone, Monica Deck, Laura Ruppert, Abena Gyasiwa, and Emily Fortier for their coordination of this work. The authors also thank IU Health, the IU Health ED clinicians, and Indiana Health Information Exchange for their participation in the study.

Author Contributions

R.L.R.: conceptualization; formal analysis, funding acquisition, investigation, methodology, project administration, supervision, validation, visualization, writing—review and editing; H.H.: conceptualization, formal analysis, investigation, methodology, project administration, validation, visualization, writing—original draft; J.H.J.: formal analysis, methodology, validation; J.T.S.: funding acquisition, resources, writing—review and editing; J.P.: data curation, resources; J.R.V.: methodology, funding acquisition, formal analysis oversight, writing—review and editing; T.K.S.: conceptualization, funding acquisition, investigation, resources, supervision, validation, visualization, writing—review and editing.

Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed and approved by the Indiana University Institutional Review Board protocol #1905749709.

^* CareWeb uses single sign-on through Cerner, typically obviating the need for clinicians to log in separately.

^* These authors contributed equally to this work.

• References

• 1 Adler-Milstein J, Embi PJ, Middleton B, Sarkar IN, Smith J. Crossing the health IT chasm: considerations and policy recommendations to overcome current challenges and enable value-based care. J Am Med Inform Assoc 2017; 24 (05) 1036-1043
• 2 Lichtner V, Baysari M. Electronic display of a patient treatment over time: a perspective on clinicians' burn-out. BMJ Health Care Inform 2021; 28 (01) e100281
• 3 Friedberg MW, Chen PG, Van Busum KR. et al. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Rand Health Q 2014; 3 (04) 1
• 4 Payne TH, Corley S, Cullen TA. et al. Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform Assoc 2015; 22 (05) 1102-1110
• 5 Bourgeois FC, Olson KL, Mandl KD. Patients treated at multiple acute health care facilities: quantifying information fragmentation. Arch Intern Med 2010; 170 (22) 1989-1995
• 6 Windle JR, Katz AS, Dow Jr JP. et al. 2016 ACC/ASE/ASNC/HRS/SCAI health policy statement on integrating the healthcare enterprise. J Am Coll Cardiol 2016; 68 (12) 1348-1364
• 7 Menachemi N, Rahurkar S, Harle CA, Vest JR. The benefits of health information exchange: an updated systematic review. J Am Med Inform Assoc 2018; 25 (09) 1259-1265
• 8 Rudin RS, Motala A, Goldzweig CL, Shekelle PG. Usage and effect of health information exchange: a systematic review. Ann Intern Med 2014; 161 (11) 803-811
• 9 The Office for the National Coordinator for Health Information Technology. Connecting health and care for the nation: a shared nationwide interoperability roadmap. Final version 1.0. Washington, DC; 2018. Accessed July 22, 2022 at: https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf
• 10 The Office of the National Coordinator for Health Information Technology. Final interoperability roadmap statements of support. HealthIT.gov. Accessed March 15, 2022 at: https://www.healthit.gov/sites/default/files/interoperability_roadmap_statements_of_support_2015-11-16_1.pdf
• 11 Centers for Medicare and Medicaid Services. MACRA: MIPS & APMs. CMS.gov. Accessed March 15, 2022 at: https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/value-based-programs/macra-mips-and-apms/macra-mips-and-apms
• 12 Holmgren AJ, Adler-Milstein J. Health information exchange in U.S. hospitals: the current landscape and a path to improved information sharing. J Hosp Med 2017; 12 (03) 193-198
• 13 Upton F. 21st Century Cures Act, H.R. 6. 114th Cong (2015–2016). Accessed March 15, 2022 at: https://www.congress.gov/bill/114th-congress/house-bill/6/text
• 14 Pronovost P, Palmer S, Johns MME. et al. Procuring interoperability: achieving high-quality, connected, and person-centered care. National Academy of Medicine. Accessed March 15, 2022 at: https://nam.edu/procuring-interoperability-achieving-high-quality-connected-and-person-centered-care/
• 15 Ruley M, Walker V, Studeny J, Coustasse A. The nationwide health information network: the case of the expansion of health information exchanges in the United States. Health Care Manag (Frederick) 2018; 37 (04) 333-338
• 16 Lin SC, Everson J, Adler-Milstein J. Technology, incentives, or both? Factors related to level of hospital health information exchange. Health Serv Res 2018; 53 (05) 3285-3308
• 17 Kibbe DC, Phillips Jr RL, Green LA. The continuity of care record. Am Fam Physician 2004; 70 (07) 1220-1223 , 1222–1223
• 18 Stack SJ, Botstein G, Mattison J. et al. Improving care: priorities to improve electronic health record usability. American Medical Association. Accessed March 15, 2022 at: https://www.ama-assn.org/sites/ama-assn.org/files/corp/media-browser/member/about-aherehr-priorities.pdf
• 19 Smith PC, Araya-Guerra R, Bublitz C. et al. Missing clinical information during primary care visits. JAMA 2005; 293 (05) 565-571
• 20 Everson J. The implications and impact of 3 approaches to health information exchange: community, enterprise, and vendor-mediated health information exchange. Learn Health Syst 2017; 1 (02) e10021
• 21 American Hospital Association. Sharing health information for treatment. TrendWatch. Accessed March 15, 2022 at: https://www.aha.org/system/files/2018-03/sharing-health-information.pdf
• 22 Furukawa MF, King J, Patel V, Hsiao CJ, Adler-Milstein J, Jha AK. Despite substantial progress In EHR adoption, health information exchange and patient engagement remain low in office settings. Health Aff (Millwood) 2014; 33 (09) 1672-1679
• 23 Holmgren AJ, Patel V, Adler-Milstein J. Progress in interoperability: measuring US hospitals' engagement in sharing patient data. Health Aff (Millwood) 2017; 36 (10) 1820-1827
• 24 Jensen LG, Bossen C. Factors affecting physicians' use of a dedicated overview interface in an electronic health record: The importance of standard information and standard documentation. Int J Med Inform 2016; 87: 44-53
• 25 Schleyer TKL, Rahurkar S, Baublet AM. et al; FHIR Development Team. Preliminary evaluation of the Chest Pain Dashboard, a FHIR-based approach for integrating health information exchange information directly into the clinical workflow. AMIA Jt Summits Transl Sci Proc 2019; 2019: 656-664
• 26 Adler-Milstein J, Wang MD. The impact of transitioning from availability of outside records within electronic health records to integration of local and outside records within electronic health records. J Am Med Inform Assoc 2020; 27 (04) 606-612
• 27 Buchanan J. Accelerating the benefits of the problem oriented medical record. Appl Clin Inform 2017; 8 (01) 180-190
• 28 Yan Q, Jiang Z, Harbin Z, Tolbert PH, Davies MG. Exploring the relationship between electronic health records and provider burnout: a systematic review. J Am Med Inform Assoc 2021; 28 (05) 1009-1021
• 29 Cantwell E, McDermott K. Making technology talk: how interoperability can improve care, drive effidcincy, and reduce waste. Healthc Financ Manage 2016; 70 (05) 70-76
• 30 Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians' clinical questions: obstacles and potential solutions. J Am Med Inform Assoc 2005; 12 (02) 217-224
• 31 Semanik MG, Kleinschmidt PC, Wright A. et al. Impact of a problem-oriented view on clinical data retrieval. J Am Med Inform Assoc 2021; 28 (05) 899-906
• 32 Koopman RJ, Kochendorfer KM, Moore JL. et al. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. Ann Fam Med 2011; 9 (05) 398-405
• 33 Kummer BR, Willey JZ, Zelenetz MJ. et al. Neurological dashboards and consultation turnaround time at an academic medical center. Appl Clin Inform 2019; 10 (05) 849-858
• 34 Khairat SS, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The impact of visualization dashboards on quality of care and clinician satisfaction: integrative literature review. JMIR Human Factors 2018; 5 (02) e22
• 35 Hartnett KP, Kite-Powell A, DeVies J. et al; National Syndromic Surveillance Program Community of Practice. Impact of the COVID-19 pandemic on emergency department visits—United States, January 1, 2019–May 30, 2020. MMWR Morb Mortal Wkly Rep 2020; 69 (23) 699-704
• 36 Venkatesh AK, Janke AT, Shu-Xia L. et al. Emergency department utilization for emergency conditions during COVID-19. Ann Emerg Med 2021; 78 (01) 84-91
• 37 Barker D, McElduff P, D'Este C, Campbell MJ. Stepped wedge cluster randomised trials: a review of the statistical methodology used and available. BMC Med Res Methodol 2016; 16 (01) 69
• 38 Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 2007; 28 (02) 182-191
• 39 Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ 2015; 350 (01) h391
• 40 Venkatesh V, Morris M, Davis G. et al. User acceptance of information technology: toward a unified view. Manage Inf Syst Q 2003; 27 (03) 425
• 41 Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4 (01) 50
• 42 Lin SC, Jha AK, Adler-Milstein J. Electronic health records associated with lower hospital mortality after systems have time to mature. Health Aff (Millwood) 2018; 37 (07) 1128-1135
• 43 Indiana Health Information Exchange. OneCare. Published 2020. Accessed September 28, 2021 at: https://www.ihie.org/onecare/#careweb
• 44 Rahurkar S, Vest JR, Finnell JT, Dixon BE. Trends in user-initiated health information exchange in the inpatient, outpatient, and emergency settings. J Am Med Inform Assoc 2021; 28 (03) 622-627
• 45 Rogers EM. Diffusion of Innovations. 3rd ed.. New York, NY: Free Press; 1983
• 46 Ratwani RM, Savage E, Will A. et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018; 25 (09) 1197-1201
• 47 Nielsen J. Usability Engineering. San Francisco, CA: Morgan Kaufmann Publishers Inc.; 1994
• 48 Fitts PM. The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 1954; 47 (06) 381-391

Publikationsverlauf

Eingereicht: 22. Juli 2022

Angenommen: 26. Januar 2023

Artikel online veröffentlicht:
28. September 2023