Keywords
Interoperability - patient safety - electronic health records
1. Background and Significance
1. Background and Significance
Electronic health records (EHRs) have been rapidly adopted by healthcare providers
in the United States with over 96% of providers currently using an EHR [[1]]. For many providers, the EHR has become the primary platform for most clinical
tasks including documenting patient information and viewing patient history, ordering
medications as well as lab and diagnostic tests, viewing results, and communicating
with other providers and patients. While EHRs are widely adopted and used extensively
for most clinical tasks, EHR technology is at varying levels of interoperability with
other health information technology (health IT) within the same provider organization
and external to the provider. For example, in a hospital setting the EHR may not be
interoperable with radiology, laboratory, or pharmacy information systems. The EHR
also may not be interoperable with health IT at external provider organizations including
EHRs at other hospitals and outpatient care facilities, and technology in support
services like laboratories.
From a technical perspective, semantic interoperability is the ability of different
information technology systems and software applications to communicate, exchange
data, and use the information that has been exchanged with the same confidence as
if that information has been generated in the same system [[2]]. From a frontline clinician perspective, interoperability can be defined as the
ability of a system to exchange electronic health information with and use electronic
health information from other systems without special effort on the part of the user
[[3]]. The criticality of EHR interoperability for improved healthcare delivery, particularly
efficiency, has been stressed by several stakeholders [[4]–[7]]. Patient safety implications associated with poor interoperability has received
considerably less attention. One study of patient safety hazards associated with computer
use by healthcare providers reported that 20% of the safety hazard reports were associated
with information transfer and about half of these were related to system integration
problems [[8]]. While this study suggests that system integration, and perhaps interoperability
impacts patient safety, it is unclear whether these safety hazards are related specifically
to the EHR and, importantly, it is unclear which clinical areas are impacted by interoperability
challenges (e.g. radiology, laboratory, pharmacy, etc.).
In this paper, we analyzed patient safety event (PSE) reports to better understand
safety hazards associated with EHRs and potential interoperability issues with other
health IT during the care process. In an effort to improve safety, most healthcare
systems have a patient safety reporting system (PSRS) in place [[9],[10]]. These systems provide a method for staff, including physicians, nurses, and technicians,
to report on safety events in their environment ranging from near misses, where harm
almost reaches a patient, to serious safety events, where a patient is harmed [[11]]. The Institute of Medicine has strongly recommended the use of these systems to
identify why patients are harmed by medical errors, and several states require the
use of a PSRS [[11]].
Patient safety event reports offer a unique lens into patient safety hazards. Details
provided in the PSE reports allow us to identify the flow of information between the
EHR and other technology (e.g. sending information or receiving information) and the
clinical process that is impacted by the interoperability challenges, such as radiology,
laboratory, or pharmacy. Conducting this type of analysis offers key insights into
the prevalence of reported interoperability challenges within different clinical areas
and the impact of these challenges on patient care. This knowledge can subsequently
be used to prioritize interoperability efforts, including identifying key system architectures
that need to be addressed and standards that need to be formulated.
2. Objectives
The purpose of this study was to understand the patient safety implications that arise
from challenges with the interoperability between EHRs and other health IT. Our objectives
were to (1) identify patient safety event reports that reflect EHR interoperability
challenges, and (2) perform a detailed analysis of these reports to understand the
health IT systems involved, the clinical care process impacted, whether the incident
occurred within or between provider organizations, and the reported severity of the
patient safety event.
3. Methods
3.1 Data Source
Data were comprised of PSE reports from the Pennsylvania Patient Safety Authority’s
Pennsylvania Patient Safety Reporting System, attained through the Institute for Safe
Medication Practices (ISMP), and a large healthcare system in the Mid-Atlantic United
States area between 2009 and 2016. Combined there were a total of 1.735 million PSE
reports included in this analysis and the reports spanned all event type categories
(e.g. falls, medication, diagnostic imaging, etc.).
3.2 The Content Structure of Patient Safety Event Reports
Patient safety event reports generally contain structured information such as the
time and site of occurrence, role of the participants (e.g. physician, nurse, or technician),
patient demographic and clinical attributes, as well as a classification of the severity
and type of event. In addition to the structured data elements, the PSE reports also
include an unstructured free-text field in which the reporter must provide a narrative
describing the event in greater detail [[12]]. Patient safety reporting systems can grow to contain tens of thousands to hundreds
of thousands of events. We analyzed thousands of PSE reports, with a focus on the
free-text narrative, to identify safety hazards associated with interoperability between
the EHR and other health IT.
3.3 Data Selection Criteria and Interoperability Classification
To identify EHR interoperability related reports we first filtered the 1.735 million
reports by those that were self-identified (i.e. user selected) as being health IT
related. This resulted in 2625 PSE reports. Using the free text narratives as the
foundation for coding, the health IT related events were manually coded by three annotators
[JP, JH, RR] with expertise in health IT to determine those events explicitly related
to interoperability. Ten percent of the data was collectively coded (e.g. each report
was coded by all three annotators) to establish inter-rater reliability. Differences
were reconciled through group discussion. Inter-rater reliability was calculated using
Fleiss’ kappa which resulted in 0.69 for the interoperability coding. The remaining
health IT related events were divided among the annotators and were individually coded
by the three annotators (e.g. each report was coded by one annotator).
For an event to be identified by the coders as being EHR and interoperability related,
the event must have explicitly identified an EHR process and interaction with another
health IT system in the narrative of the event report. Computerized provider order
entry (CPOE) is considered a component of the EHR. Other health IT systems include
any system that is intended to interface with the EHR such as radiology systems (e.g.
image viewing and results reporting systems), laboratory systems, pharmacy systems,
devices (e.g. glucometer), billing and registration systems, and other EHRs (e.g.
from external provider organizations or departments within the provider organization).
Examples of EHR interoperability and non-interoperability reports are provided in
[Table 1].
Table 1
Examples of interoperability and non-interoperability associated patient safety event
reports.
|
Interoperability Related PSE Report
|
Non-Interoperability Related PSE Report
|
|
Example 1: Recurring problem of pathology reports not interfacing with EHR from lab
documentation tool. Biopsy ordered and performed. Report not sent to the patient’s
chart.
|
Example 3: Pt was scheduled for surgery. Pre-op orders were written at 9am but not
released in EHR by the ordering prescriber. Delayed releasing of pended orders and
administration of the medications led to delay in start of procedure by 1 hour.
|
|
Example 2: Physician placed electronic medication orders in EHR for drug. Medication
orders were not in pharmacy system. IT notified- interface issue. Medication orders
did not reach pharmacy system. Patient missed 1 dose of medication.
|
Classifying each of the PSE reports necessitated certain assumptions. We assumed clinical
competency, meaning providers know the right actions to take unless explicitly stated
otherwise (e.g. providers correctly entered orders into the appropriate sections of
the EHR), and that electronic systems were being used unless explicitly stated otherwise
(e.g. paper charts) and were functioning as intended. The EHR and other health IT
systems could be from the same or different vendors.
Reports that described a challenge with two aspects or components of the same EHR
product were excluded from analysis. We excluded PSE reports that occurred during
EHR downtime, network outages, or connectivity failures. Ambiguously written narratives
and those involving patient portals were also excluded. In addition, reports that
described events explicitly caused by human error, such as wrong patient selection,
and those that involved devices that did not explicitly connect to an EHR system (e.g.
bed alarms, medication dispensers) were excluded from analysis.
Ultimately, 209 PSE reports were determined to be related to interoperability. The
review process is shown in [Figure 1].
Fig. 1 Overview of the process to identify EHR interoperability related safety events.
3.4 Coding EHR Interoperability Associated Reports
With the 209 interoperability events identified, we coded the reports to identify
the clinical areas involved, the aspect of the clinical process in which the issue
occurred, whether events occurred within or between provider sites, and the reported
harm score.
Clinical Areas of Interoperability Related Events
In our coding, interoperability related PSE reports were analyzed by the following
clinical areas:
-
Medication events occurred between the EHR (CPOE), the pharmacy health IT systems,
and the electronic medication administration record (EMAR).
-
Laboratory events occurred between the EHR (CPOE), laboratory systems (e.g. LIS),
and other EHR systems or components of the EHR.
-
Radiology events occurred between the EHR (CPOE), radiology systems, (e.g. imaging
device, PACS, RIS, IDX), and other EHRs or components of the EHR.
-
Device events occurred between devices (e.g. glucometers, medication scanners) and
the EHR.
-
The interoperability events that did not fit into the medication, laboratory, radiology,
or device categories were coded as “other”. Examples of “other” interoperability events
include the EHR and the billing system or the EHR and the dietary services system.
Sending/Receiving Information and Interoperability Related Events
Each interoperability related event was coded to determine whether the event included
the EHR sending information to another health IT system (e.g. medication order; EHR
(CPOE) sends information to the pharmacy system) or receiving information from another
health IT system (e.g. radiology results; radiology system sends results/images to
the EHR).
Provider Sites and Interoperability Related Events
The health IT systems involved in the PSE reports were coded to determine whether
the interoperability challenge was within a provider organization, between a provider
organization and an external organization, or unclear. If an event report described
two systems that were within the same provider organization (e.g. hospital EHR to
hospital pharmacy system) then it was coded as within. If an event report described
two systems that were at different provider organizations (e.g. outpatient care facility
to hospital) then it was coded as external. Events where the provider sites could
not be determined were coded as unclear.
Harm Scores of Interoperability Related Events
The person reporting the PSE provides a self-reported severity level or harm score
of the safety hazard which is categorized as an A through I level of harm (left side
of [Table 2]). We analyzed the harm scores of the events identified as EHR interoperability related
and organized the original harm scores into four categories (right side of [Table 2]).
Table 2
Reported severity scores and categories used in analysis.
|
Original Reported PSE Severity Score
|
New Categories of Severity Scores
|
|
A. Unsafe Condition (Non Event)
|
Unsafe Condition
|
|
B1. Near Miss – No Harm – Didn’t Reach Patient – Caught By Chance
|
Didn’t Reach Patient – No Harm
|
|
B2. Near Miss – No Harm – Didn’t Reach Patient – Active Recovery By Caregivers
|
|
C. No Harm – Reached Patient – No Monitoring Required
|
Reached Patient – No Harm
|
|
D. No Harm – Reached Patient – Monitoring Required
|
|
E. Harm – Temporary Harm – Intervention Needed
|
Patient Harm
|
|
F. Harm – Temporary Harm – Hospitalization Needed
|
|
G. Harm – Permanent Harm
|
|
H. Harm – Permanent Harm – Intervention Required to Sustain Life
|
|
I. Death
|
4. Results
Two hundred nine (8%) PSE reports of the 2625 health IT reports were determined to
be related to interoperability between the EHR and another health IT system. The interoperability
related PSE reports were further analyzed to examine clinical areas, systems sending/receiving
information, provider site, and harm score as outlined below.
Clinical Areas of Interoperability Related PSE Reports
Medication events comprised 60 (29%) of the interoperability related reports, followed
by laboratory (N = 55, 26%), radiology (N = 43, 21%), other (N = 29, 14%) and device
(N = 22, 11%).
Process Areas of Interoperability Related PSE Reports
Of the five categories of EHR interoperability events, medication, radiology, and
laboratory involve information being sent from the EHR to systems in these areas (e.g.
orders) and information being sent back to the EHR (e.g. results or tasks to complete).
These three clinical process areas account for 158 (76%) of the interoperability related
reports. From this subset, only 51 (32%) of reports described a hazard that occurred
when EHR (CPOE) sent information to another clinical health IT system. The majority
of these reports, 107 (68%), involved information being sent from another health IT
system into the EHR. [Figure 2] illustrates the directional network relationships between these clinical areas and
the EHR.
Fig. 2 EHR and interoperability with other health IT systems.
Provider Sites and Interoperability Related PSE Reports
As seen in [Table 3] below, 167 (80%) reports incorporated sufficient information for the coders to understand
the relationship between the two health IT systems involved, either within the same
provider organization (N = 130, 62%) or from one provider to an external provider
organization (N = 37, 18%). Forty-two reports (20%) did not include enough information
for the research team to determine within/external provider organizations.
Table 3
Count and percent of PSE reports describing interoperability within and to external
provider organizations.
|
Count of Interoperability Related PSE Reports
|
Percent of Total Interoperability PSE Reports
|
|
Within Provider Organization
|
130
|
62%
|
|
To an External Provider Organization
|
37
|
18%
|
|
Cannot be Determined
|
42
|
20%
|
Harm Scores of Interoperability Related PSE Reports
We analyzed the patient harm score for the 209 interoperability related events as
seen in [Figure 3]. Overall, “unsafe conditions” accounted for 38 (18%) of all interoperability events,
“didn’t reach patient – no harm” 55 events (26%), “reached patient – no harm” 111
events (53%), and “patient harm” 5 events (2%).
A majority of medication (N = 42, 70%), laboratory (N = 33, 60%), and radiology (N
= 22, 51%) system related events reached the patient. Device and other interoperability
related PSE reports primarily described an unsafe hazard that was caught before reaching
the patient; 15 (68%) of device events and 17 (59%) of other events never reached
the patient
Fig. 3 Harm scores of interoperability related PSE reports identified by clinical areas
5. Discussion
In this study, we examined self-identified health IT PSE reports and coded events
that were related to interoperability issues with an EHR and other health IT systems
during the care process. Interoperability related events were further analyzed by
clinical areas, process areas, within/external provider organizations, and severity
level of the reports. Our analysis demonstrates that interoperability failures are
involved in patient safety hazards, some of which lead to patient harm.
In our analysis approximately 8% of self-identified health IT PSE reports were related
to interoperability issues, consistent with previous literature [[13]]. The results demonstrate that medication, laboratory, and radiology events accounted
for over three-quarters of all interoperability related events identified with medication
events being the most frequently reported. The majority of interoperability safety
events had to do with the EHR receiving information from other systems and not the
EHR sending information. Surprisingly, most interoperability challenges were within
a provider organization as opposed to challenges sending or receiving information
to external provider organizations. Finally, many interoperability related events
reached the patient, however, most events did not result in harm to the patient.
Our analysis highlights important areas for focused interoperability groundwork and
the knowledge gleaned from the PSE report analysis can be used to prioritize interoperability
efforts. This study suggests that medication events should be of central focus, followed
by laboratory and radiology. For large provider organizations, efforts should be focused
on interoperability between the EHR and other information technology within the organization.
The results highlight important areas of focus for future research. It is important
to understand the results presented here in the context of overall EHR activity. Currently,
it is unclear as to what proportion of radiology-, medication-, and laboratory-related
actions comprise all EHR actions and therefore it is difficult to gauge if one clinical
process is particularly over or under-represented in the patient safety event data.
In addition, as health IT becomes more pervasive it will be important to develop patient
safety event reporting systems that capture health IT interactions to further safety
improvements, without burdening the reporter. This might include automated capture
of the health IT systems that are active during a safety hazard that is being reported.
There are limitations to our PSE analysis. Patient safety event reports are a self-reported
data source and often these data have not been reviewed for accuracy of the event
that is being described. We analyzed reports self-identified as health IT related;
however, there may be many more health IT related events that were not identified
as such, though previous research suggests that the sensitivity of self-identification
is high. A 2014 analysis of overall health information technology adverse events in
two reporting systems concluded that approximately 98.8% of the event report data
was correctly coded as “not health IT related”[[14]]. Corroborating this finding in our data by identifying health IT related events
that were not explicitly identified by the reporter is a focus of future work. Our
analysis is also limited by the details provided in each event report. In addition,
patient safety reports do not capture all of the safety hazards in healthcare and
are subject to the biases of the frontline staff entering the reports. These data
serve as one source of insight into patient safety hazards.
6. Conclusions
The most common EHR interoperability challenges, as identified through the analysis
of patient safety event reports, involved the EHR interfacing with pharmacy, laboratory,
and radiology systems, respectively. Interoperability efforts should prioritize these
areas, given their clinical importance, and providers should recognize that many interoperability
challenges involve the EHR interfacing with other systems within their own organization.
Clinical Relevance Statement
Clinical Relevance Statement
Our research identifies pharmacy, laboratory, and radiology as critical areas to focus
EHR interoperability efforts. Health information technology stakeholders should not
only focus on interoperability between healthcare provider organizations, but should
also recognize the importance of interoperability within a provider organization between
the EHR and other health information technology.
Multiple choice question
When examining EHR interoperability with other health information technology what
clinical process is associated with the most patient safety event reports?
-
Radiology
-
Laboratory
-
Devices
-
Medication
Correct Answer: D (Medication)
Explanation: Our analysis of patient safety event reports describing EHR interoperability
with other health information technology systems and was analyzed by clinical process.
The results show that EHR interoperability with pharmacy was the most frequently occurring
safety reports followed by laboratory and radiology.
