Electronic Health Record Adoption and Nurse Reports of Usability and Quality of Care: The Role of Work Environment

Abstract

Background Despite evidence suggesting higher quality and safer care in hospitals with comprehensive electronic health record (EHR) systems, factors related to advanced system usability remain largely unknown, particularly among nurses. Little empirical research has examined sociotechnical factors, such as the work environment, that may shape the relationship between advanced EHR adoption and quality of care.

Objective The objective of this study was to examine the independent and joint effects of comprehensive EHR adoption and the hospital work environment on nurse reports of EHR usability and nurse-reported quality of care and safety.

Methods This study was a secondary analysis of nurse and hospital survey data. Unadjusted and adjusted logistic regression models were used to assess the relationship between EHR adoption level, work environment, and a set of EHR usability and quality/safety outcomes. The sample included 12,377 nurses working in 353 hospitals.

Results In fully adjusted models, comprehensive EHR adoption was associated with lower odds of nurses reporting poor usability outcomes, such as dissatisfaction with the system (odds ratio [OR]: 0.75; 95% confidence interval [CI]: 0.61–0.92). The work environment was associated with all usability outcomes with nurses in better environments being less likely to report negatively. Comprehensive EHRs (OR: 0.83; 95% CI: 0.71–0.96) and better work environments (OR: 0.47; 95% CI: 0.42–0.52) were associated with lower odds of nurses reporting fair/poor quality of care, while poor patient safety grade was associated with the work environment (OR: 0.50; 95% CI: 0.46–0.54), but not EHR adoption level.

Conclusion Our findings suggest that adoption of a comprehensive EHR is associated with more positive usability ratings and higher quality of care. We also found that—independent of EHR adoption level—the hospital work environment plays a significant role in how nurses evaluate EHR usability and whether EHRs have their intended effects on improving quality and safety of care.

Keywords

electronic health records - nursing - health care quality - work environment

Background and Significance

The Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 spurred the rapid adoption of electronic health records (EHRs) in U.S. hospitals.[1] In 2008, 13% of hospitals in the country were using a basic EHR system; by 2015, nearly 90% of hospitals had one in place.[2] Over the same 7-year period, the number of hospitals implementing comprehensive EHR systems that have more advanced functionalities, such as clinical decision support, grew from less than 2 to 40%.[2] While some evidence suggests that higher levels of EHR adoption produce safer and higher quality care,[3] [4] [5] [6] [7] a growing body of literature has documented unintended negative consequences of the technology,[8] [9] including disruptions in clinical workflows and usability concerns that may negatively affect quality of care and patient safety.[10] [11] [12] These system issues may disproportionally impact the work of nurses who are among the highest EHR users in hospitals.

Nurses use EHR systems for documentation, medication administration, clinical monitoring and decision-making, and coordination of patient care. Few studies have examined the usability of EHR systems from the nurse perspective, including overall satisfaction, as well as how these systems influence care activities and patient outcomes. With the exception of one statewide study in Texas,[13] the vast majority of evidence from U.S. hospitals conducted following the HITECH Act comes from single institutions.[14] [15] [16] Taken together, the findings of these studies are mixed. While nurses have identified positive aspects of EHRs,[9] [13] [14] [16] many nurses remain dissatisfied with their use, find them burdensome and time consuming, and express doubts in the ability of the systems to improve patient care.[15] [17] These inconclusive findings may be attributable not only to the level of EHR adoption, but also to differences in the organizational and clinical environments into which the systems are implemented. Some evidence suggests that hospitals with better work environments may be more likely to adopt EHR systems of greater complexity;[18] however, it remains unknown how the work environment may affect the relationship between EHR use and outcomes.

Sociotechnical theory suggests that EHR systems cannot be designed or implemented successfully without thoughtful consideration of the “fit” of the technology with the people providing care and the existing context of care delivery, including the work environment.[19] [20] [21] [22] [23] Indeed, reviews of EHR outcome studies often cite this lack of fit as one of the most likely reasons for undesirable results following system introduction.[24] [25] [26] [27] Most of these reviews, however, draw upon qualitative reports in single institutions, while large-scale, empirical studies of the environmental factors that facilitate or impede EHR adoption and use remain virtually absent from the literature.

A substantial body of research has identified features of work environments that support nurses in their practice and are conducive to high-quality patient care, including adequate nurse staffing and resources, promotion of nurse autonomy, involvement of nurses in administrative decision-making, strong nursing leadership, and teamwork among staff.[28] [29] [30] [31] While Hessels and coworkers[4] considered the work environment as a potential confounder of the relationship between EHR adoption and patient satisfaction, no studies to our knowledge have considered the work environment as a primary factor in the relationship between EHR adoption, end-user usability assessments, and the quality and safety of patient care.

Objective

The objective of this study was to examine the independent and joint effects of comprehensive EHR system adoption and the work environment on nurse reports of system usability, including satisfaction and effectiveness, and nurse-reported quality of care and safety.

Methods

Study Design

This study was a secondary analysis of three linked data sources, including (1) the 2015 to 2016 RN4CAST-US nurse survey,[32] (2) the 2015 American Hospital Association (AHA) Annual Survey Database,[33] and (3) the 2015 AHA Healthcare Information Technology (IT) Database.[34] Data were linked using a common hospital identifier. The analysis included hospitals that had at least 10 nurse survey respondents to provide reliable estimates of the work environment[32] and were represented in the AHA Annual Survey and IT databases. The response rate for the AHA Annual Survey exceeds 75% each year,[35] while the response rate to the 2015 AHA Healthcare IT Survey was 56%.[2]

Data Sources

The RN4CAST-US survey[32] was mailed to the home addresses of a random sample of 231,000 registered nurses (RNs) obtained from state licensure lists across four U.S. states, including California, Pennsylvania, New Jersey, and Florida. The RN4CAST-US survey instrument included questions regarding nurse demographics, patient record system usability, and validated measures of the work environment, staffing, quality of care and patient safety. Nurses who reported working in hospitals were also asked to provide the name of their employing hospital. This made it possible to match nurses to hospitals and allowed for the aggregation of individual nurse responses to the hospital level. Ultimately, we obtained responses from 12,377 staff nurses working in direct patient care in 353 hospitals in the four states for which we also had AHA Annual Survey and IT data. On average, each hospital included in our analysis was represented by 35 nurse respondents.

The 2015 AHA Annual Survey provided information related to structural characteristics of hospitals, including teaching status, size, and ability to perform high-technology procedures, core based statistical area (CBSA)—a census-based measure of population density—and ownership. The 2015 AHA IT Database provided information on the degree to which hospitals have implemented a set of 31 individual functionalities related to the following six areas: (1) electronic clinical documentation, (2) results viewing, (3) computerized provider order entry (CPOE) for laboratory and radiology tests, medication, consultation requests, and nursing orders, (4) decision support, (5) bar coding, and (6) other functionalities (e.g., telemedicine). Each of the functionalities are coded on a 6-point scale of: (1) fully implemented across all units; (2) fully implemented in at least one unit; (3) beginning to implement in at least one unit; (4) have resources to implement in the next year; (5) do not have resources but considering implementing; and (6) not in place and not considering implementation.

Measures

EHR Adoption Level

Using the AHA IT Database, we classified hospitals into two groups based on their reported EHR adoption level: (1) basic system or less or (2) comprehensive system. Following definitions provided by the Office of the National Coordinator for Health Information Technology,[2] hospitals were classified as having a basic EHR if the following were either fully implemented on at least one clinical unit or across all units: (1) electronic clinical documentation of demographics, problem lists, medication lists, and discharge summaries, (2) electronic laboratory, radiologic and diagnostic test results, and (3) CPOE. Given most hospitals have some form of EHR technology,[2] we combined hospitals with a basic EHR system with those that did not meet the requirements of a basic system. Comprehensive EHR systems were defined as the presence of the three core components of a basic system (i.e., electronic clinical documentation of demographics, problem lists, medication lists, and discharge summaries; electronic laboratory, radiologic and diagnostic test results, and CPOE), as well as 14 additional functionalities that have been implemented fully across all units.[2]

Work Environment

The hospital work environment was measured using the Practice Environment Scale of the Nursing Work Index (PES-NWI), a 31-item instrument that has been endorsed by the National Quality Forum as a patient safety performance measure.[31] [36] The PES-NWI is one of the most commonly used and reliable instruments available to conduct comparisons of organizational work factors among hospital nurses.[37] [38] Nurses were asked to indicate the degree to which various organizational features are present in their practice setting. A hospital-level measure of the work environment was created by first aggregating individual nurse responses for each of the five established PES-NWI subscales. The subscales include nurse participation in hospital affairs (9 items), nursing foundations for quality care (10 items), nurse manager ability, leadership, and support of nurses (5 items), staffing and resource adequacy (4 items), and collegial nurse–physician relations (3 items).[31] A PES composite score for each hospital was created by calculating the mean of the five hospital-level subscales. For this analysis, we used the PES composite score for each hospital to categorize work environments as follows: better (≥75th percentile), mixed (26–74th percentile), and poor (≤25th percentile).[29] [39]

EHR Usability

Nurse respondents rated their satisfaction with their current patient record system on a 4-point Likert scale ranging from “very satisfied” to “very dissatisfied.” Nurses were also asked to indicate their level of agreement with a set of seven items adapted from previous studies[40] [41] that measured their evaluations of the system's effectiveness in their daily work. On a 4-point Likert scale ranging from “strongly agree” to “strongly disagree,” nurses reported on: ability to quickly access patient information, interference of the system in the provision of care, ease of use, trust in the accuracy of patient assessment and medication information, the system's ability to support work efficiency, and the ability of the system to easily share information between team members. Responses were dichotomized into two categories (i.e., satisfied/dissatisfied; agree/disagree) to facilitate interpretation of the results. Respondents were also asked to respond to an item inquiring whether nurses were involved in choosing or modifying the system.

Nurse-Reported Quality of Care and Patient Safety

Respondents were also asked to rate the quality of care provided in their work setting on a 4-point Likert scale of excellent, good, fair, or poor. Responses of fair/poor, and excellent/good were combined for the analysis. Using an item adapted from the Agency for Healthcare Research and Quality's Patient Safety Culture Survey,[42] nurses assigned an overall grade for their unit on patient safety as: A: excellent; B: very good; C: acceptable; D: poor; or F: failing. Grades of C, D, or F were defined as a poor safety grade.

Covariates

Hospital structural characteristics were derived from AHA Annual Survey data. These characteristics have been shown to be related to quality of care as well as EHR adoption.[2] Teaching status was classified into three categories based on the ratio of medical trainees to beds: none, minor (<1:4), and major (>1:4). Size was classified as small (≤100 beds), medium (101–250 beds), and large (>250 beds). Hospitals with high technology capability were identified as facilities that provide services for open-heart surgery, organ transplantation, or both. CBSA type was classified as metropolitan, micropolitan, or rural. Ownership was categorized as government nonfederal, nonprofit, and for-profit. State was specified as California, Pennsylvania, New Jersey, or Florida. Nurse characteristics obtained from the RN4CAST-US survey included age, sex, years of RN experience, and education level (bachelor's degree in nursing or higher) and were included as covariates because of their associations with EHR acceptance.[13] [14] [24] Using nurse responses from RN4CAST-US survey, we also accounted for nurse staffing by creating a hospital-level measure that was derived by dividing the average number of patients present on each unit during the last shift by the average number of RNs present on the last shift.

Data Analysis

Descriptive statistics were calculated to assess differences in hospital and nurse characteristics by EHR adoption level. EHR usability measures and nurse-reported patient outcomes were examined descriptively for nurses who worked in hospitals with and without comprehensive EHRs and by work environment classification. Robust logistic regression models that accounted for the clustering of nurses within hospitals were used to examine the effect of comprehensive EHR system adoption and the work environment on outcomes, first separately, then jointly, and finally in a fully adjusted model that accounted for other hospital and nurse characteristics. Odds ratios (ORs) created by exponentiating the regression coefficients are presented for ease of interpretation, along with 95% confidence intervals (CIs). An interaction term between comprehensive EHR system adoption and work environment was tested, but was not statistically significant. Analyses were conducted with SAS version 9.3, (SAS Institute, Inc, Cary, North Carolina, United States) and results were considered statistically significant at p < 0.05.

Results

Of the 353 hospitals in the sample, 157 (44%) had adopted a comprehensive EHR system, while 196 (56%) had a basic system or less. Among hospitals with a basic EHR system or less, 53 (15%) had less than a basic system in place. [Table 1] shows the characteristics of nurses and hospitals in the sample by level of EHR adoption. Of note, higher proportions of nurses working in hospitals with comprehensive EHR systems held a bachelor's degree in nursing or higher (58.9 vs. 55.2%, p < 0.001) and, on average, had more years of RN experience (18.9 vs. 17.9, p < 0.001) compared with nurses working in hospitals with basic systems or less. Among hospital characteristics, significant variation in EHR adoption level was observed by teaching status and ownership (both p < 0.001). There were no statistically significant differences in the adoption of comprehensive EHRs by hospital size, technology status, state, or CBSA.

Table 1 Distribution of nurse and hospital characteristics by level of electronic health record (EHR) adoption

		EHR adoption level
Nurse characteristics	All (n = 12,377 nurses)	Basic EHR or less (n = 6,133)	Comprehensive EHR (n = 6,244)	p-Value[a]
Age, mean (SD)	46.8 (12.3)	46.7 (12.2)	46.9 (12.3)	0.43
Female, n (%)	11,158 (90.4%)	5,487 (89.6%)	5,671 (91.1%)	0.004
Highest nursing degree, baccalaureate, n (%)	7,034 (57.0%)	3,369 (55.2%)	3,665 (58.9%)	<0.001
Years of RN experience, mean (SD)	18.4 (12.9)	17.9 (12.7)	18.9 (13.0)	<0.001
Hospital characteristics	All ( n  = 353 hospitals)	Basic EHR or less ( n  = 196)	Comprehensive EHR ( n  = 157)	p -Value [a]
Size, n (%)
≤ 100 beds	16	9 (56.3%)	7 (43.8%)	0.99
101–250 beds	127	70 (55.1%)	57 (44.9%)
> 250 beds	210	117 (55.7%)	93 (44.3%)
Teaching status, n (%)
Nonteaching	148	66 (33.7%)	82 (52.2%)	<0.001
Minor teaching	165	114 (58.2%)	51 (32.5%)
Major teaching	40	16 (8.2%)	24 (15.3%)
Technology capability, n (%)
Low	136	78 (57.4%)	58 (42.7%)	0.58
High (performs open heart surgery and/or organ transplants)	217	118 (54.4%)	99 (45.6%)
State, n (%)
California	138	81 (58.7%)	57 (41.3%)	0.46
Florida	95	55 (57.9%)	40 (42.1%)
New Jersey	41	19 (46.3%)	22 (53.7%)
Pennsylvania	79	41 (51.9%)	38 (48.1%)
Core-based statistical area (CBSA), n (%)
Metropolitan	342	188 (55.0%	154 (45.0%)	0.32
Micropolitan	8	5 (62.5%)	3 (37.5)
Rural	3	3 (100.0%)	0 (0%)
Ownership, n (%)
Government, nonfederal	32	15 (46.9%)	17 (53.1%)	<0.001
Nonprofit	265	134 (50.6%)	131 (49.4%)
For-profit	56	47 (83.9%)	9 (16.1%)

Abbreviations: RN, registered nurse; SD, standard deviation.

Note: Percentages may not add to 100 due to rounding and small amounts of missing data (<5%) on nurse characteristics. A total of 53 hospitals had adopted less than a basic EHR.

^a p-values generated from chi-square for all variables, except for CBSA where Fisher's exact test was used.

Nurse assessments of the system usability and quality of care by level of EHR adoption are displayed in [Table 2]. Overall, 25.1% of nurses expressed dissatisfaction with the EHR system. The percentage of nurses who were dissatisfied was slightly lower in hospitals with comprehensive EHRs compared with hospitals with a basic system or less (21.6 vs. 28.7%, p < 0.001). Across all outcomes studied, the percentage of nurses reporting poor usability outcomes with the EHR was significantly lower in hospitals with comprehensive systems. Notably, over half (55.4%) of the surveyed nurses reported that EHRs interfered with patient care, while nearly one-third (31.9%) reported that they were not easy to use and did not help them to do their work in an efficient way (32.2%). Nearly half (48.5%) of survey respondents reported that nurses were not involved in choosing or modifying the system; however, this percentage was slightly lower in hospitals adopting comprehensive systems (46.4 vs. 50.7%, p < 0.001). Compared with hospitals with basic EHR systems or less, fewer nurses in hospitals with comprehensive systems reported poor quality of care (9.9 vs. 14.0%, p < 0.001) and a poor patient safety grade (26.5 vs. 30.3%, p < 0.001).

Table 2 Nurse assessments of electronic health record (EHR) system usability and quality of care in hospitals by level of EHR adoption

		N (%) of nurses agreeing with statement who work in hospitals with:
Outcomes	All (n = 12,377)	Basic EHR or less (n = 6,133)	Comprehensive EHR (n = 6,244)	p-Value[a]
Not satisfied with the system	2,597 (25.1%)	1,462 (28.7%)	1,135 (21.6%)	<0.001
The systems do not make it easy to access a patient's clinical data quickly	2,034 (19.7%)	1,204 (23.7%)	830 (15.8%)	<0.001
The systems interfere with the provision of patient care	5,668 (55.4%)	2,848 (56.7%)	2,820 (54.2%)	0.009
The systems are not easy to use	3,290 (31.9%)	1,809 (35.7%)	1,481 (28.2%)	<0.001
I do not trust the accuracy of the patient assessment data documented in the systems	1,802 (17.5%)	955 (18.8%)	847 (16.2%)	<0.001
I do not trust the accuracy of the medication information in the systems	1,145 (11.1%)	617 (12.2%)	528 (10.1%)	<0.001
The systems do not help me to do my work in an efficient way	3,321 (32.2%)	1,778 (35.1%)	1,543 (29.4%)	<0.001
The systems do not make it easy to share information in a timely way with other members of the health care team	2,107 (20.5%)	1,248 (24.6%)	859 (16.4%)	<0.001
Nurses were not involved in choosing (or modifying) the patient record system	3,183 (48.5%)	1,643 (50.7%)	1,540 (46.4%)	<0.001
Quality of care on unit is fair or poor	1,244 (11.9%)	721 (14.0%)	523 (9.9%)	<0.001
Poor overall unit grade on patient safety (C,D, or F)	2,980 (28.4%)	1,565 (30.3%)	1,415 (26.5%)	<0.001

Note: Percentages in the “All” column may not align with overall sample size due to missing data. Sample sizes range from 6,565 to 10,495.

^a p-values generated from chi-squares.

[Table 3] demonstrates statistically significant differences (p < 0.001) in each of the studied outcomes related to EHR usability and nurse-reported quality of care and safety across work environment types. Nearly 40% of nurses working in hospitals with poor work environments reported dissatisfaction with the record system compared with less than 20% of nurses working in hospitals with better environments. Also of note, nearly half of nurses (45.6%) working in poor environments reported that the EHR system did not help them to work efficiently, compared with one-quarter (25.2%) of nurses in better environments. In hospitals with poor work environments, over two-thirds (67.6%) of nurses reported that nurses were not involved in choosing or modifying the record system compared with about one-third (34.4%) of nurses working in better conditions.

Table 3 Nurse assessments of electronic health record (EHR) system usability and quality of care in hospitals by work environment type

		N (%) of nurses agreeing with statement who work in hospitals with a work environment that is:
Outcomes	All (n = 12,377)	Poor (n = 2,256)	Mixed (n = 6,214)	Better (n = 3,907)	p-Value[a]
Not satisfied with the patient record system	2,597 (25.1%)	730 (38.6%)	1,282 (24.7%)	585 (17.9%)	<0.001
The systems do not make it easy to access a patient's clinical data quickly	2,034 (19.7%)	557 (29.7%)	1,001 (19.3%)	476 (14.6%)	<0.001
The systems interfere with the provision of patient care	5,668 (55.4%)	1,146 (61.9%)	2,874 (55.8%)	1648 (51.2%)	<0.001
The systems are not easy to use	3,290 (31.9%)	816 (43.5%)	1,657 (31.9%)	817 (25.1%)	<0.001
I do not trust the accuracy of the patient assessment data documented in the systems	1,802 (17.5%)	522 (27.8%)	883 (17.0%)	397 (12.2%)	<0.001
I do not trust the accuracy of the medication information in the systems	1,145 (11.1%)	313 (16.7%)	579 (11.2%)	253 (7.8%)	<0.001
The systems do not help me to do my work in an efficient way	3,321 (32.2%)	854 (45.6%)	1,649 (31.8%)	818 (25.2%)	<0.001
The systems do not make it easy to share information in a timely way with other members of the health care team	2,107 (20.5%)	559 (29.8%)	1,055 (20.4%)	493 (15.2%)	<0.001
Nurses were not involved in choosing (or modifying) the patient record system	3,183 (48.5%)	819 (67.6%)	1,641 (50.5%)	723 (34.4%)	<0.001
Quality of care on unit is fair or poor	1,244 (11.9%)	433 (23.0%)	631 (12.0%)	180 (5.5%)	<0.001
Poor overall unit grade on patient safety (C,D, or F)	2,980 (28.4%)	889 (46.8%)	1,538 (29.1%)	553 (16.7%)	<0.001

Note: Percentages in the “All” column may not align with overall sample size due to missing data. Sample sizes range from 6,565 to 10,495.

^a p-values generated from chi-squares.

[Table 4] shows the ORs estimating the effects of EHR adoption level and work environment type on each of the study outcomes. The first column of the table presents models where the effects of EHR adoption level and work environment on outcomes were estimated separately. In these bivariate models, both predictors had a statistically significant effect on outcomes with adoption of a comprehensive EHR and better work environments being associated with lower likelihoods of unfavorable usability and nurse-reported quality and safety outcomes except for two cases where EHR adoption level was not significant at the p < 0.05 level (system interference with the provision of patient care and nurses were not involved in choosing/modifying the record system). In jointly estimated models (second column), both main effects were slightly attenuated but maintained statistical significance in most outcomes. In three cases where EHR adoption level was significant when it was estimated separately, the effect was no longer significant once work environment was accounted for (i.e., trust in the accuracy of patient assessment data, trust in the accuracy of medication information, and poor patient safety grade).

Table 4 Odds ratios (OR) indicating the effects of a comprehensive electronic health record (EHR) and work environment on nurse assessments of EHR system usability and quality of care (n = 12,377)

	Estimated separately	Estimated jointly	Fully adjusted
Outcome	OR (95% CI)	OR (95% CI)	OR (95% CI)
Not satisfied with the system
Comprehensive EHR	0.68 (0.55–0.84)[a]	0.74 (0.61–0.90)[b]	0.75 (0.61–0.92)[b]
Work environment	0.59 (0.51–0.68)[a]	0.61 (0.52–0.70)[a]	0.67 (0.58–0.77)[a]
System does not make it easy to access a patient's clinical data quickly
Comprehensive EHR	0.60 (0.49–0.74)[a]	0.64 (0.53–0.78)[a]	0.65 (0.53–0.79)[a]
Work environment	0.64 (0.55–0.73)[a]	0.66 (0.58–0.76)[a]	0.73 (0.64–0.83)[a]
The systems interfere with the provision of patient care
Comprehensive EHR	0.90 (0.81–1.01)	0.93 (0.84–1.04)	0.97 (0.87–1.08)
Work environment	0.81 (0.74–0.88)[a]	0.81 (0.75–0.88)[a]	0.84 (0.77–0.92)[a]
Systems are not easy to use
Comprehensive EHR	0.71 (0.59–0.84)[a]	0.75 (0.64–0.89)[a]	0.78 (0.66–0.93)[b]
Work environment	0.66 (0.59–0.75)[a]	0.68 (0.60–0.76)[a]	0.76 (0.67–0.86)[a]
I do not trust the accuracy of the patient assessment data documented in the system
Comprehensive EHR	0.83 (0.71–0.98)[c]	0.90 (0.78–1.05)	0.94 (0.81–1.10)
Work environment	0.60 (0.54–0.67)[a]	0.61 (0.54–0.68)[a]	0.64 (0.57–0.72)[a]
I do not trust the accuracy of the medication information in the systems
Comprehensive EHR	0.81 (0.68–0.96)[c]	0.87 (0.73–1.03)	0.85 (0.70–1.02)
Work environment	0.65 (0.57–0.73)[a]	0.66 (0.58–0.74)[a]	0.69 (0.60–0.80)[a]
Systems do not help me to do my work in an efficient way
Comprehensive EHR	0.77 (0.65–0.91)[b]	0.82 (0.71–0.96)[b]	0.85 (0.73–1.00)[c]
Work environment	0.64 (0.57–0.72)[a]	0.65 (0.58–0.73)[a]	0.71 (0.63–0.80)[a]
Systems do not make it easy to share information in a timely way with other members of the health care team
Comprehensive EHR	0.60 (0.49–0.73)[a]	0.64 (0.53–0.77)[a]	0.64 (0.54–0.77)[a]
Work environment	0.65 (0.57–0.75)[a]	0.67 (0.59–0.77)[a]	0.76 (0.67–0.86)[a]
Nurses were not involved in choosing/modifying the patient record system
Comprehensive EHR	0.84 (0.69–1.03)	0.94 (0.80–1.11)	1.01 (0.86–1.18)
Work environment	0.50 (0.45–0.56)[a]	0.51 (0.45–0.57)[a]	0.59 (0.53–0.67)[a]
Quality of care on unit is fair or poor
Comprehensive EHR	0.67 (0.56–0.82)[a]	0.77 (0.67–0.88)[a]	0.83 (0.71–0.96)[c]
Work environment	0.44 (0.40–0.49)[a]	0.45 (0.41–0.50)[a]	0.47 (0.42–0.52)[a]
Poor patient safety grade (C, D, or F)
Comprehensive EHR	0.83 (0.71–0.97)[c]	0.94 (0.85–1.03)	0.98 (0.89–1.09)
Work environment	0.48 (0.44–0.52)[a]	0.48 (0.45–0.52)[a]	0.50 (0.46–0.54)[a]

Abbreviations: CBSA, core-based statistical area; CI, confidence interval; RN, registered nurse.

Notes: Odds ratios are from robust logistic regression models adjusted for hospital characteristics (size, teaching status, high-technology capability, ownership, CBSA type, state, and nurse staffing level), nurse characteristics (age, sex, highest level of education—baccalaureate in nursing or higher, and years of RN experience), and the clustering of nurses within hospitals. The odds ratio associated with the work environment can be interpreted as the difference in the odds of the nurse-reported outcome between hospitals with “better versus mixed” environments, and between hospitals with “mixed versus poor” environments.

^a p < 0.001.

^b p < 0.01.

^c p < 0.05.

The last column of [Table 4] presents the fully adjusted models that include EHR adoption level and work environment in addition to hospital and nurse characteristics. In models related to EHR usability outcomes, adoption of a comprehensive EHR was associated with lower odds of nurses reporting dissatisfaction with the system (OR: 0.75; 95% CI: 0.61–0.92), that the system does not make it easy to access clinical data (OR: 0.65; 95% CI: 0.53–0.79), the system is not easy to use (OR: 0.78; 95% CI: 0.66–0.93), that the system does not help in performing work efficiently (OR: 0.85; 95% CI: 0.73–1.00), and that the system does not make it easy to share information with other members of the health care team (OR: 0.64; 95% CI: 0.54–0.77). The work environment was associated with all EHR usability outcomes with nurses in hospitals with better environments being less likely than nurses in mixed environments to report: overall dissatisfaction with their current record system environments (OR: 0.67; 95% CI: 0.58–0.77), that the system does not make it easy to access clinical data (OR: 0.73; 95% CI: 0.64–0.83), system interference with the provision of patient care (OR: 0.84; 95% CI: 0.77–0.92), that the system is not easy to use (OR: 0.76; 95% CI: 0.67–0.86), distrust in the accuracy of patient assessment data (OR: 0.64; 95% CI: 0.57–0.72) and medication information (OR: 0.69; 95% CI: 0.60–0.80), that the system does not help in performing work efficiently (OR: 0.71; 95% CI: 0.63–0.80), and that the system does not make it easy to share information with other health care team members (OR: 0.76; 95% CI: 0.67–0.86). Nurses in better work environments were also nearly half as likely as those in mixed environments to report that nurses were not involved in choosing/modifying the system (OR: 0.59; 95% CI: 0.53–0.67). By extension, these models also imply that nurses working in better environments were significantly less likely than those working in poor environments to report these negative outcomes. For example, nurses working in better environments were 55% ([1–0.672] × 100) less likely than nurses in poor environments to report overall dissatisfaction with the EHR. Among the hospital and nurse characteristics included in the models as controls, we noted that a different set of these covariates was associated with each outcome; however, hospital ownership and nurse experience were two of the covariates that were consistently statistically significant. In fully adjusted models of nurse-reported quality and patient safety outcomes, the odds of nurses reporting fair/poor quality of care were significantly lower in hospitals that had adopted a comprehensive EHR system compared with those that did not (OR: 0.83; 95% CI: 0.71–0.96) and in hospitals with better as opposed to mixed work environments (OR: 0.47; 95% CI: 0.42–0.52). Compared with nurses in mixed environments, nurses working in hospitals in better environments were half as likely to report a poor patient safety grade (OR: 0.50; 95% CI: 0.46–0.54). Adoption of comprehensive EHR system did not have a significant effect on safety grade after the work environment was taken into account.

Discussion

Our findings suggest that adoption of a comprehensive EHR is associated with more positive usability ratings by nurses and higher nurse-reported quality of care. We also found that—independent of EHR adoption level—the work environment plays a significant role in how nurses evaluate EHR usability. EHR adoption level was associated with 6 of the 11 outcomes studied, while the work environment was associated with all 11. This result implies that adoption of advanced EHR systems and the quality of the work environment both play significant roles in the delivery of high quality care, and also that the work environment may play a significant role in the success of EHR system adoption.

To our knowledge, this is one of the first studies to examine the concurrent effects of EHR adoption level and the hospital work environment on usability and quality outcomes. In models where EHR adoption level and work environment were considered separately, both variables had significant associations with study outcomes. However, when both factors were considered simultaneously in fully adjusted models, we noted that—in most cases—the effect of EHR adoption level on outcomes was moderated and sometimes rendered insignificant after including the work environment. This moderation effect was especially notable for the outcome of poor/failing patient safety grade, and suggests that the work environment may play a more important role in the delivery of safe patient care than the type of EHR system. A similar phenomenon was also observed by Hessels and coworkers[4] who noted that one element of the work environment—staffing and resource adequacy—moderated the relationship between EHR adoption level and patient satisfaction in a sample of New Jersey hospitals. Other research has documented that face-to-face communication between nurses and physicians decreases following EHR implementation,[43] which suggests that having a strong culture of interprofessional teamwork—another component of good work environments—becomes even more important with the introduction of electronic record systems. Taken together, these findings provide empirical evidence for the consideration of the hospital work environment as a critical factor in the implementation and use of hospital EHR systems.

With a sample of over 12,000 nurses, our study represents one of the largest to examine nurse satisfaction with EHR systems and reports of their clinical utility. Although nurses working in hospitals with comprehensive EHR systems and in the highest-rated work environments reported the most favorable outcomes, our study highlights a significant need for improvement across hospitals at all levels of EHR adoption. Overall, we found that 25% of nurses are dissatisfied with their current record systems while similarly high percentages reported usability issues. Overall, nearly 50% of nurses in our sample reported that they were not involved in the selection or modification of the EHR. This is a concerning finding given that nurses are one of the primary groups of EHR end-users and may provide an explanation for the significant proportions of nurses reporting dissatisfaction and poor usability. Across all studied EHR usability outcomes, nurses working in better work environments were significantly less likely to assign negative evaluations of the system compared with nurses working in less favorable environments, which suggests that strengthening organizational structures, such as nurse leadership and engagement in every stage of EHR adoption from system selection to the post-implementation period, may serve as potential points of intervention. This set of findings is aligned with prior research demonstrating that successful EHR implementation is more likely when EHRs are perceived by clinicians to be helpful to them in their practice and to have positive effects on patient outcomes.[9] [44] [45] A concerted effort is also needed to improve hospital work environments: nearly one-third of nurses in our sample reported working in fair or poor conditions.

As nearly all U.S. hospitals have adopted some form of EHR technology, the focus now turns toward the adoption of advanced functionalities that relate to interoperability, performance measurement, and patient engagement.[46] Evidence is mounting that use of these advanced functionalities—that are often included in comprehensive EHR systems—is needed to achieve the EHR's desired effects.[3] [4] [7] Our study adds to this evidence base by demonstrating that nurses are less likely to report poor quality in hospitals with comprehensive EHR systems, and that comprehensive systems are more likely to support nurses in the complex tasks of patient care delivery. This finding provides valuable evidence to policymakers and to hospitals enrolled in the Medicare and Medicaid EHR Incentive Program by demonstrating that investment in advanced systems may translate into the delivery of higher quality of care. As participants in this program, eligible hospitals must attest to regularly scheduled advancements in the functionality of their EHR system to avoid payment penalties. Future work should explore associations between EHR system capabilities, usability, and clinical patient outcomes, such as mortality and readmissions.

Limitations

Our cross-sectional analysis limits the ability to draw causal inferences about the relationships we observed. We also acknowledge the potential bias that may be introduced due to unmeasured variables. For example, we were unable to account for the number of years that the EHR system was in place for each hospital. The more experience that clinicians have with EHR systems may be associated with more favorable observations of their effectiveness,[13] [16] as well as better patient outcomes.[47] Few rural hospitals were included in our sample which limits conclusions that can be made about EHR use in those hospitals. Recent research suggests a widening divide in adoption of comprehensive EHRs between urban and rural hospitals.[48] Our sample was limited to hospitals in California, New Jersey, Pennsylvania, and Florida, which may limit the generalizability of our findings; however, the hospitals located in these four states represent over 20% of all acute care hospitals nationally, and discharge over 25% of all patients in the country. Further, a previous comparison of RN4CAST-US and 2017 Current Population Survey data found that hospital RNs in these four states are similar to hospital RNs nationally on characteristics such as age and education.[49] Finally, our categorization of basic EHR system or less includes hospitals that vary widely in their adoption levels, ranging from a basic system implemented in all units, a basic system implemented on some units, or less than a basic system. However, the primary aim of this study was to examine whether the adoption of comprehensive EHR systems with advanced capabilities—the desired goal for all U.S. hospitals—was associated with more favorable reports of EHR usability and quality of care.

Conclusion

EHRs, particularly those with advanced capabilities, have been widely promoted as a means to decrease health care costs, improve efficiency, and optimize patient safety. Our study findings suggest that adoption of a comprehensive EHR system is associated with greater nurse satisfaction with the system, more favorable reports of the system's usability, and higher quality of care. The value of EHR may not be fully realized, however, unless we understand the context in which they most effectively function. Independent of the EHR adoption level, we found that the work environment is highly associated with nurses' ratings of the usefulness of EHR systems, and most importantly that the quality of the work environment may determine the extent to which comprehensive EHR systems have their intended impact on quality and safety. The best patient outcomes may be achieved by concurrently implementing comprehensive EHR systems and improving hospital work environments.

Clinical Relevance Statement

One of the leading reasons for poor outcomes following EHR adoption may be the lack of consideration for how the system will interface with end-users and the existing organizational structure. Adoption of a comprehensive EHR system is associated with greater satisfaction and usability ratings by nurses, as well as higher nurse-reported quality of care. The best outcomes may be achieved by concurrently implementing comprehensive EHR systems and improving hospital work environments.

Multiple Choice Questions

1. The findings of this study suggest that the most favorable outcomes related to EHR adoption may be achieved by implementing a comprehensive EHR system and improving what organizational feature?

   • The hospital work environment.

   • Facilities.

   • Room design.

   • Lighting.

   Correct Answer: The correct answer is option a, the hospital work environment. Our findings suggest that adoption of a comprehensive EHR system is associated with greater nurse satisfaction with the system, more favorable reports of the system's usability, and higher quality of care. Independent of the EHR adoption level, we found that the work environment is highly associated with nurses' ratings of the usefulness of EHR systems. The quality of the work environment may determine the extent to which comprehensive EHR systems have their intended impact on health care quality and safety.

2. What is an example of a work environment-focused intervention that could potentially improve nurse perceptions of electronic health record (EHR) system usability?

   • Hire fewer nurses.

   • Provide handouts on the benefits of the system for nurses to read.

   • Strengthen nurse leadership and engagement in all phases of EHR adoption.

   • Conduct seminars on communication on every unit.

   Correct Answer: The correct answer is option c, strengthen nurse leadership and engagement in all phases of EHR adoption. Sociotechnical theory suggests that EHR systems cannot be implemented successfully without thoughtful consideration of the “fit” of the technology with the people providing care and the existing context of care delivery, including the work environment. Our study found that the hospital work environment was highly associated with nurses' ratings of the usefulness of EHR systems. Key features of the work environment include adequate nurse staffing and resources, promotion of nurse autonomy, involvement of nurses in administrative decision-making, strong nursing leadership, and teamwork.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

This study was reviewed, and received exempt status, by the University of Pennsylvania Institutional Review Board.

• References

• 1 Adler-Milstein J, Jha AK. HITECH Act drove large gains in hospital electronic health record adoption. Health Aff (Millwood) 2017; 36 (08) 1416-1422
• 2 Henry J, Pylypchuk Y, Searchy T, Patel V. Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008–2015. ONC Data Brief 35. Washington, DC: Office of the National Coordinator for Health Information Technology; 2016
• 3 Adler-Milstein J, Everson J, Lee SY. EHR adoption and hospital performance: time-related effects. Health Serv Res 2015; 50 (06) 1751-1771
• 4 Hessels A, Flynn L, Cimiotti JP, Bakken S, Gershon R. Impact of heath information technology on the quality of patient care. Online J Nurs Inform 2015; 19: 19
• 5 Walker-Czyz A. The impact of an integrated electronic health record adoption on nursing care quality. J Nurs Adm 2016; 46 (7–8): 366-372
• 6 Furukawa MF, Eldridge N, Wang Y, Metersky M. Electronic health record adoption and rates of in-hospital adverse events. J Patient Saf 2016;
• 7 Jarvis B, Johnson T, Butler P. , et al. Assessing the impact of electronic health records as an enabler of hospital quality and patient satisfaction. Acad Med 2013; 88 (10) 1471-1477
• 8 Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care--an interactive sociotechnical analysis. J Am Med Inform Assoc 2007; 14 (05) 542-549
• 9 Gephart S, Carrington JM, Finley B. A systematic review of nurses' experiences with unintended consequences when using the electronic health record. Nurs Adm Q 2015; 39 (04) 345-356
• 10 Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 246-250
• 11 Meeks DW, Smith MW, Taylor L, Sittig DF, Scott JM, Singh H. An analysis of electronic health record-related patient safety concerns. J Am Med Inform Assoc 2014; 21 (06) 1053-1059
• 12 Howe JL, Adams KT, Hettinger AZ, Ratwani RM. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018; 319 (12) 1276-1278
• 13 McBride S, Tietze M, Hanley MA, Thomas L. Statewide study to assess nurses' experiences with meaningful use-based electronic health records. Comput Inform Nurs 2017; 35 (01) 18-28
• 14 Higgins LW, Shovel JA, Bilderback AL. , et al. Hospital nurses' work activity in a technology-rich environment: a triangulated quality improvement assessment. J Nurs Care Qual 2017; 32 (03) 208-217
• 15 Schenk EC, Mayer DM, Ward-Barney E, Estill P, Goss L, Shreffler-Grant J. RN perceptions of a newly adopted electronic health record. J Nurs Adm 2016; 46 (03) 139-145
• 16 Ward MM, Vartak S, Schwichtenberg T, Wakefield DS. Nurses' perceptions of how clinical information system implementation affects workflow and patient care. Comput Inform Nurs 2011; 29 (09) 502-511
• 17 Topaz M, Ronquillo C, Peltonen LM. , et al. Nurse informaticians report low satisfaction and multi-level concerns with electronic health records: results from an international survey. AMIA Annu Symp Proc 2017; 2016: 2016-2025
• 18 Lippincott C, Foronda C, Zdanowicz M, McCabe BE, Ambrosia T. The relationship between magnet designation, electronic health record adoption, and Medicare meaningful use payments. Comput Inform Nurs 2017; 35 (08) 385-391
• 19 Carayon P. Sociotechnical systems approach to healthcare quality and patient safety. Work 2012; 41 (Suppl. 01) 3850-3854
• 20 Harrison MI, Henriksen K, Hughes RG. Improving the health care work environment: a sociotechnical systems approach. Jt Comm J Qual Patient Saf 2007; 33 (11, Suppl): 3-6
• 21 Carayon P, Bass E, Bellandi T, Gurses A, Hallbeck S, Mollo V. Socio-technical systems analysis in health care: a research agenda. IIE Trans Healthc Syst Eng 2011; 1 (01) 145-160
• 22 D'Agostino F, Zega M, Rocco G, Luzzi L, Vellone E, Alvaro R. Impact of a nursing information system in clinical practice: a longitudinal study project. Ann Ig 2013; 25 (04) 329-341
• 23 Gurses AP, Ozok AA, Pronovost PJ. Time to accelerate integration of human factors and ergonomics in patient safety. BMJ Qual Saf 2012; 21 (04) 347-351
• 24 Strudwick G, McGillis Hall L, Nagle L, Trbovich P. Understanding nurses' perceptions of electronic health record use in an acute care hospital setting. Stud Health Technol Inform 2016; 225: 795
• 25 Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood) 2011; 30 (03) 464-471
• 26 Boonstra A, Versluis A, Vos JF. Implementing electronic health records in hospitals: a systematic literature review. BMC Health Serv Res 2014; 14: 370
• 27 Black AD, Car J, Pagliari C. , et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011; 8 (01) e1000387
• 28 McHugh MD, Kelly LA, Smith HL, Wu ES, Vanak JM, Aiken LH. Lower mortality in magnet hospitals. Med Care 2013; 51 (05) 382-388
• 29 Aiken LH, Clarke SP, Sloane DM, Lake ET, Cheney T. Effects of hospital care environment on patient mortality and nurse outcomes. J Nurs Adm 2008; 38 (05) 223-229
• 30 Kutney-Lee A, Germack H, Hatfield L. , et al. Nurse engagement in shared governance and patient and nurse outcomes. J Nurs Adm 2016; 46 (11) 605-612
• 31 Lake ET. Development of the Practice Environment Scale of the Nursing Work Index. Res Nurs Health 2002; 25 (03) 176-188
• 32 Sloane DM, Smith HL, McHugh MD, Aiken LH. Effect of changes in hospital nursing resources on improvements in patient safety and quality of care: a panel study. Med Care 2018; 56 (12) 1001-1008
• 33 American Hospital Association. AHA Annual Survey Database. Available at: http://www.ahadata.com/aha-annual-survey-database-asdb/ . Accessed December 3, 2018
• 34 American Hospital Association. AHA Healthcare IT Database. Available at: http://www.ahadata.com/aha-healthcare-database/ . Accessed December 3, 2018
• 35 American Hospital Association. Data collection methods. Available at: http://www.ahadata.com/data-collection-methods/ . Accessed December 3, 2018
• 36 National Quality Forum. 0206: Practice Environment Scale - Nursing Work Index (PES-NWI) (composite and five subscales). 2015. Available at: http://www.qualityforum.org/QPS/020 . Accessed December 5, 2018
• 37 Swiger PA, Patrician PA, Miltner RSS, Raju D, Breckenridge-Sproat S, Loan LA. The Practice Environment Scale of the Nursing Work Index: an updated review and recommendations for use. Int J Nurs Stud 2017; 74: 76-84
• 38 Warshawsky NE, Havens DS. Global use of the Practice Environment Scale of the Nursing Work Index. Nurs Res 2011; 60 (01) 17-31
• 39 McHugh MD, Ma C. Hospital nursing and 30-day readmissions among Medicare patients with heart failure, acute myocardial infarction, and pneumonia. Med Care 2013; 51 (01) 52-59
• 40 Sockolow PS, Weiner JP, Bowles KH, Lehmann HP. A new instrument for measuring clinician satisfaction with electronic health records. Comput Inform Nurs 2011; 29 (10) 574-585
• 41 Otieno OG, Toyama H, Asonuma M, Kanai-Pak M, Naitoh K. Nurses' views on the use, quality and user satisfaction with electronic medical records: questionnaire development. J Adv Nurs 2007; 60 (02) 209-219
• 42 Agency for Healthcare Research and Quality. Hospital Survey on Patient Safety Culture. Rockville, MD. Available at: http://www.ahrq.gov/sops/quality-patient-safety/patientsafetyculture/hospital/index.html . Accessed December 5, 2018
• 43 Taylor SP, Ledford R, Palmer V, Abel E. We need to talk: an observational study of the impact of electronic medical record implementation on hospital communication. BMJ Qual Saf 2014; 23 (07) 584-588
• 44 Sockolow PS, Liao C, Chittams JL, Bowles KH. Evaluating the impact of electronic health records on nurse clinical process at two community health sites. 11th International Congress on Nursing Informatics, June 23–27, 2012, Montreal, Canada. NI 2012; 2012:381
• 45 Bristol AA, Nibbelink CW, Gephart SM, Carrington JM. Nurses' use of positive deviance when encountering electronic health records-related unintended consequences. Nurs Adm Q 2018; 42 (01) E1-E11
• 46 Adler-Milstein J, Embi PJ, Middleton B, Sarkar IN, Smith J. Crossing the health IT chasm: considerations and policy recommendations to overcome current challenges and enable value-based care. J Am Med Inform Assoc 2017; 24 (05) 1036-1043
• 47 Lin SC, Jha AK, Adler-Milstein J. Electronic health records associated with lower hospital mortality after systems have time to mature. Health Aff (Millwood) 2018; 37 (07) 1128-1135
• 48 Adler-Milstein J, Holmgren AJ, Kralovec P, Worzala C, Searcy T, Patel V. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide. J Am Med Inform Assoc 2017; 24 (06) 1142-1148
• 49 Aiken LH, Sloane DM, Barnes H, Cimiotti JP, Jarrín OF, McHugh MD. Nurses' and patients' appraisals show patient safety in hospitals remains a concern. Health Aff (Millwood) 2018; 37 (11) 1744-1751