Keywords system usability scale - health information exchange - pharmacists - pharmacies
Background and Significance
Background and Significance
Health information exchange (HIE) provides multiple health care professionals (HCPs) access to and sharing of patient data from multiple electronic health records (EHRs), facilitating safer, more tailored patient care[1 ] across health care organizations. At the same time, almost 90% of all patients in the United States live within 5 miles of a community pharmacy; thus, community pharmacists are one of the most accessible HCPs for patients.[2 ] However, community pharmacists often do not have routine access to the same patient HIE data as other health care organizations, leading to potential medication discrepancies and suboptimal patient care.[3 ] When community pharmacists have access to patient data from other HCPs, they are able to identify and prevent prescription errors, identify more discordant medications and resolve more medication therapy-related problems.[4 ]
[5 ]
[6 ]
[7 ]
[8 ]
[9 ] This leads to improved patient care and decreased hospital admissions.[5 ]
[7 ] Roberts et al found when community pharmacists have access to laboratory results, progress notes, medication diagnoses, and allergy information, 39% of medication therapy management consults can be completed without needing to contact other HCPs, increasing the efficiency of health care delivery.[10 ] Further, there is evidence that community pharmacists want access to more patient data. For instance, in the state of Indiana, Sethman et al found that community pharmacists wanted access to specific HIE data, including updated medication orders, progress notes, and laboratory results.[11 ]
Other HCPs, including community pharmacists in some states, have “read-only” access to local health-system EHRs.[4 ]
[6 ]
[12 ] This limited access provides community pharmacists with some of the information they need but also leads to workflow inefficiencies and duplicative documentation.[4 ] One way to increase access to patient data for community pharmacists could be through participation in regional HIEs. HIEs facilitate the sharing of patient data among multiple HCPs across varying health systems involved in a patient's care. Patients visit multiple health care facilities, and collaboration among all HCPs involved in a patient's care is of utmost importance for improved patient outcomes.[4 ] Optimal utilization of HIEs can facilitate better communication among all HCPs through easy access to all patient data, including clinical documents, in one place.[13 ]
[14 ]
[15 ]
[16 ]
[17 ] Despite the accessibility and variety of services community pharmacists provide to patients, community pharmacists rarely have HIE access.[15 ]
[17 ] There is a general paucity of literature describing HIE integration for community pharmacies,[15 ] but in the few instances where community pharmacists do have access to HIE, access is traditionally only available via logging into an external platform with an interface not integrated with the pharmacy's existing software.[4 ] Separate navigation to several external systems is inefficient and can cause workflow disruptions, hindering the usability of HIE for pharmacy workflows.[4 ]
[18 ]
[19 ]
It is critical for community pharmacists and pharmacy support staff (i.e., pharmacy technicians) to have access to HIE data from within their existing clinical information systems, allowing pharmacy professionals to expand the level of care they provide to patients. Pharmacists and pharmacy technicians should be involved in all phases of the design of an interface for HIE data access to ensure that their needs are met before implementation.[20 ]
[21 ]
[22 ] From our literature review, however, pharmacy professionals have not been adequately involved in the development and design of an HIE interface for use in community pharmacies.
Objective
Our objective was to conduct a formative usability evaluation of an HIE interface prototype with community pharmacists and pharmacy technicians. This will then inform strategies to improve the HIE interface design for future implementation in community pharmacies.
Methods
Health Information Exchange Access and Interface Design
For this research, we collaborated with the Indiana Health Information Exchange (IHIE) and PioneerRx. We designed and evaluated a novel prototype interface (referred to as the “HIE-Pioneer mock-up” throughout) intended to enable community pharmacy professionals to access statewide HIE data from within their existing software system in the future. IHIE is one of the most robust HIEs in the United States,[23 ] and PioneerRx is a pharmacy dispensing software already used by over 5,000 community pharmacies across the United States.[24 ] The PioneerRx user interface has approximately 15 years of design language incorporated into the application. The design is centered around (and licensed from) current Microsoft Windows and Office best practices and design language. This HIE-Pioneer mock-up includes links to two key IHIE applications: (1) an existing IHIE application “Clinical Data Search,”[25 ] which is a “search” bar for all patient data available within IHIE's patient data repository and (2) our team's adaptation of the existing IHIE application (HealthDart), which we named “Cardiovascular Risk Profile.” The latter is a dashboard of selected laboratory results and immunizations applicable to cardiovascular conditions commonly encountered by community pharmacy professionals, such as diabetes, dyslipidemia, and hypertension.[6 ]
[26 ] We designed the prototype so both HIE applications were accessible from three different screens within the PioneerRx system, which were reported by the software vendor as key work areas for pharmacy teams. For example, one HIE access point was an “Action” menu to find HIE content as well as related, existing software applications. From here, pharmacy professionals can take several related, clinical actions (i.e., access prescription drug monitoring programs) that were already implemented in Pioneer Rx. The other two original HIE access points we chose were also contextually related to routine pharmacy workflow tasks. In our prototype, these chosen access points also followed consistent design language used throughout PioneerRx.
Findings from previous literature, such as pharmacy teams lack of access to patient data[27 ] as well as our own preliminary research, such as the need for integration with existing software systems and an intuitive interface, were incorporated into our initial HIE-Pioneer mock-up development.[11 ] Additionally, we incorporated design input from a multidisciplinary team of pharmacists, information technology specialists, and a usability expert to build an interface that would efficiently provide access to external patient data for pharmacy teams in a way that was consistent with their current workflow. The HIE-Pioneer mock-up was developed as a clickable, interactive portable document format to provide a low-cost, but realistic mechanism for formative usability evaluation. Screenshots of the pharmacy teams' existing pharmacy dispensing software, PioneerRx, were utilized to visualize where link outs to IHIE applications would be integrated into PioneerRx (see [Supplementary Fig. S1 ], available in the online version). Screenshots of ‘Clinical Data Search’ ([Fig. 1 ]) and the new ‘Cardiovascular Risk Reduction Profile’ ([Fig. 2 ]) were also utilized. These could be accessed via three different places within the HIE-Pioneer mock-up.
Fig. 1 Clinical Data Search. “Clinical Data Search” is an existing Indiana Health Information Exchange application. “Clinical Data Search” is a “search” bar health care professionals can use to search for any patient data available within IHIE's patient data repository.
Fig. 2 Cardiovascular (CV) Risk Profile. The CV Risk Profile is an adaption of an existing Indiana Health Information Exchange application, HealthDart. Designed specifically for use by pharmacy professionals, the CV Risk Profile provides access to laboratory data and vaccine information applicable to cardiovascular conditions such as diabetes, hypertension, and dyslipidemia.
Two community pharmacists who were ineligible to participate in this study provided feedback on our initial designs. The HIE-Pioneer mock-up was iteratively redesigned by the team over the course of 4 months and four iterations prior to conducting the formative usability evaluation.
Study Design
We conducted a scenario-based, formative usability evaluation study of the HIE-Pioneer mock-up with community pharmacists and pharmacy technicians. Usability sessions were scheduled for 60 minutes and followed a standardized protocol including four scenarios. Usability sessions were intentionally conducted with participants the first time they interacted with the HIE-Pioneer mock-up for two reasons: (1) so that our study could identify the most common usability problems prior to initial implementation of a working HIE interface; and (2) because community pharmacies routinely hire new pharmacists and pharmacy technicians who will need to utilize HIE software quickly with minimal training support. This study was approved by the Indiana University Institutional Review Board. This manuscript was informed by STARE-HI guidelines.[28 ]
Usability Testing Scenario Development
We developed four standardized clinical patient scenarios representing day-to-day community pharmacy workflow[21 ] for usability testing. The same standardized, fictious patient was utilized for all four scenarios, and scenarios were altered slightly for pharmacists versus pharmacy technicians to align with their specific roles (see [Supplementary Appendix S1 ], available in the online version). Scenarios were primarily developed by a community pharmacist researcher and reviewed with other team members with expertise in human factors engineering and health services research. Scenarios covered four critical aspects of community pharmacy workflow: drug utilization review (DUR) for a new prescription, DUR during a routine medication refill/medication synchronization, medication therapy management (encompassing comprehensive medication reviews and targeted medication reviews) and immunization counseling. These scenarios were reviewed and refined based on feedback from two community pharmacists not included in the main study.
Pharmacy Selection and Participant Recruitment
All community pharmacies (n = 25) within the Community Pharmacy Enhanced Services Network of Indiana[29 ] that utilize PioneerRx as their pharmacy software vendor were eligible to participate. Of these, three pharmacies were purposefully sampled to ensure variation in geographic location across the state, high level of engagement in practice transformation initiatives and infrastructure variation, such as number of staff members employed. We contacted four pharmacies until three interested locations were selected. Up to three pharmacists and three technicians currently employed at each pharmacy were eligible to participate in usability testing, for an anticipated sample size of up to 18 participants. For this formative usability evalutation, we targeted this sample size as a sample size of 10 to 15 participants has been reported in previous literature to identify 90 to 95% of major usability problems when using the think aloud technique.[30 ]
[31 ]
[32 ]
[33 ] When less than three pharmacists or technicians were employed by the pharmacy site, all eligible individuals were invited to participate. When more than three pharmacists or technicians were employed, we used further purposive sampling to ensure variation in participants' role, such as pharmacist manager versus staff pharmacist. Each pharmacy site received $500 total compensation. Individual participants received no compensation.
Data Collection
A standardized script was utilized for verbal consent with each participant prior to conducting the usability evaluation. Data were collected from participants in a private space separate from their usual pharmacy workflow. A pharmacist researcher, mentored by a usability expert, moderated all usability evaluations. The moderator read a standardized introduction to the usability evaluation. Participants also watched a 1-minute-long video on “think aloud”[31 ] prior to beginning the usability evaluation. Participants were asked to “think aloud” and verbalize all thoughts on the prototype, both positive and negative.[32 ] Participants used the moderators' laptop computer to complete all aspects of the usability evaluation. The moderator reminded participants to think aloud if they were quiet during the evaluation. Participants' screen actions were video recorded on Zoom for Healthcare (Zoom Video Communications, San Jose, California, United States) while they interacted with the HIE-Pioneer mock-up. Each participant had the list of four scenarios printed out in front of them and verbally stated when they had completed a scenario. After they indicated they had completed the final scenario, the moderator asked any clarifying and/or debrief questions before ending the recording. Participants' screen interactions with the prototype were recorded to collect usability data. Each participant was then asked to use a laptop to complete and submit the system usability scale (SUS).[34 ] These responses were collected and managed using REDCap electronic data capture tool.[35 ]
[36 ] Each participant had up to 1 hour to complete four applicable patient scenarios and an unlimited amount of time to complete the 10 item SUS questionnaire.
Video recordings from each usability evaluation were analyzed to assess the following: types of usability problems, time on task, task success rates, and interface application utilization. Similar to other published literature,[37 ] usability problems were identified primarily by a research trainee who was mentored by a usability expert. The usability expert and trainee individually reviewed a subset of videos independently to identify usability problems, then met to compare and discuss their findings. Videos were reviewed in this manner until the trainee demonstrated consistent skill in identifying usability problems. At that point, the trainee independently analyzed the remaining videos. Each video recording was reviewed to identify usability problems and note other widely accepted usability attributes, such as learnability, efficiency, and perceived satisfaction.[38 ] Each usability problem was categorized into one of three domains: (1) IHIE integration with PioneerRx, (2) Clinical Data Search, and (3) Cardiovascular Risk Profile. The number and type of participant(s) affected by each error across all sites was also recorded. Usability problems in each domain were organized into one of five categories, listed in order of priority for HIE-Pioneer design modifications: (1) ideally must be fixed prior to actual use and full-scale implementation; (2) high impact on usability/low effort to address; (3) high impact/high effort; (4) low impact/low effort; and (5) low impact/high effort.[39 ] All findings were prioritized by the full team and included input from the software vendors on level of expected effort. Usability problems that fell into category “1,” indicating highest priority to address, included problems that impacted over four HCPs and/or presented a major safety concern.[39 ]
Time on task was determined by reviewing timestamps on each video from the start of each scenario to the end. Start times were defined as the moment a participant moved their mouse after starting each scenario. End times were defined as when the participant verbally stated or confirmed they had completed the scenario OR stopped moving their mouse, whichever occurred first. Time on task measurements were averaged for each participant type and scenario using Microsoft Excel.
Task success rates and interface application utilization rates were also gathered from analysis of the usability evaluation videos. Task success rates were determined by reviewing each scenario for completeness and considered “successful” if the participant accessed at least one application and demonstrated confidence that they found the intended patient information for the given scenario. Examples of intended information participants were expected to access are included in [Supplementary Appendix S1 ] (available in the online version) with scenario descriptions. The rates of interface application utilization were calculated by noting whether the participant accessed the Clinical Data Search and/or the CV Risk Profile for each scenario. SUS scores were calculated[34 ] using SPSS (v.28 IBM, Armonk, NY, United States) and correlated to the Bangor Adjective Rating scale[40 ] to measure user satisfaction.
After data collection and analyses of usability evaluations were complete, a subset of pharmacists (n = 2) and pharmacy technicians (n = 1) were invited to give further input via our evidenced-based quality improvement advisory panel.[41 ] Panelists were purposefully sampled to ensure variation in pharmacy employment location, role at the pharmacy, age, gender, and general attitudes about the HIE-Pioneer mock-up displayed in usability evaluations. Panelists engaged in two virtual, 2-hour long advisory panel sessions to review our study findings and inform future prototype improvements.
Results
Participant Demographics
Sixteen participants, eight pharmacists and eight pharmacy technicians, across three different community pharmacy sites completed the study. Sites were geographically dispersed across Indiana. All participants identified as White and most identified as female (85.7% pharmacists and 83.3% pharmacy technicians). Pharmacy site-level characteristics are described in [Table 1 ].
Table 1
Site-level characteristics for participating community pharmacies, (n = 3)
Pharmacies affected n (%)
Type of community pharmacy
Independent (<four locations)
2 (66.7)
Chain (four or more locations)
1 (33.3)
Location of community pharmacy, n (%)
Urban
1 (33.3)
Rural
2 (66.7)
Integrated platforms available, n (%)
CHIRP[a ]
1 (33.3)
OutcomesMTM[b ]
1 (33.3)
eCare Plans[c ]
2 (66.7)
INSPECT [d ]
1 (33.3)
Abbreviations: CHIRP, Children and Hoosiers Immunization Registry Program; eCare, electronic care; INSPECT, Indiana Prescription Monitoring Program; MTM, Medication Therapy Management.
a CHIRP refers to the statewide database for immunization history.
b OutcomesMTM is a medication therapy management vendor utilized by community pharmacy teams.
c eCare planning platforms are one mechanism for pharmacists to document and submit interventions made.
d INSPECT is the Indiana prescription drug monitoring program.
Usability of Health Information Exchange-Pioneer Mock-up
A total of 23 distinct usability problems were identified through usability evaluations with pharmacy professionals. Usability problems by domain and frequency are presented in [Table 2 ]. Five high impact usability problems (i.e., usability problems categorized as a “1”) were identified for the HIE-Pioneer interface mock-up, and the team-developed CV Risk Profile will be expanded upon throughout the results and discussion. High-impact usability problems with the HIE-Pioneer interface mock-up design specifically included (1) difficulty finding HIE tools within the existing Pioneer workflow; (2) expressed frustration with the multiple clicks needed to access information; and (3) desire for the design of the HIE interface mock-up to be more consistent with features in their existing workflow, such as the utilization of “tabs.” High-impact usability findings with the team-developed CV Risk Profile included (1) ambiguity with the HIE label name resulting in limited use and (2) date labels for laboratory values needing clarification. Participants also demonstrated difficulty navigating and “logging out” of the Clinical Data Search application. One pharmacist noted, “if I were new to the program it [the HIE-Pioneer mock-up] would be a lot of clicks to find the information… data is there but it is a lot of clicks to get to it.” A pharmacy technician noted, “[This HIE-Pioneer mock-up] would be really useful.” The HIE-Pioneer mock-up received an average satisfaction score of 69 on the SUS[34 ] (range 0–100, where 100 is the best) from pharmacy professionals. Pharmacists generally reported higher satisfaction with the HIE-Pioneer mock-up than pharmacy technicians with an average score of 74 compared with 65, respectively ([Table 3 ]). Task success (i.e., completion of scenario-based tasks and identification of information needed) and frequency of prototype access are reported in [Table 4 ]. In advisory panel sessions, solutions to address high-impact findings were discussed ([Table 3 ]), such as the ambiguity in the “Cardiovascular (CV) Risk Profile” label name limiting use. Some suggestions for other names for the tool label included “Metabolic Dashboard” and “Cardiovascular Labs and Vaccines.” Participants within the advisory panel agreed that changing the label name to something more self-explanatory may increase use. Overall pharmacy professionals who accessed the CV Risk Profile liked the layout, with one professional stating “[This] is much more user friendly than CHIRP [Indiana's State Immunization Registry].”
Table 2
Usability problems identified across all community pharmacy sites (n = 3) and participants (n = 16)
No. of HCPs (RPh, tech)
1. HIE interface prototype access within PioneerRx
a. Unable to find HIE tools within existing PioneerRx workflow[a ]
8 (3, 5)
b. Expressed frustration about multiple clicks needed to access HIE tools[a ]
7 (4, 3)
c. Prefers use of Pioneer “tabs” feature to access HIE data [no general consensus amongst HCPs on how to organize the data using tabs]
6 (3, 3)
d. Desires easy access to patient appointment information within HIE tools
2 (0, 2)
2. Cardiovascular (CV) risk profile
a. Ambiguity with “CV risk profile” label name limited use[a ]
7 (4, 3)
b. Dates for laboratory and vaccine data warrant clarification[a ]
5 (2, 3)
c. Fill History dates warrant clarification
1 (1, 0)
d. Difficulty “logging out” of CV risk profile
3 (1, 2)
e. Desired integrated vaccine snapshot data into existing PioneerRx Vaccine tabs
2 (2, 0)
f. Wants vaccine information to be bidirectionally exchanged[a ]
1 (1, 0)
g. Anticipated access to list of disease states and medications within this function
1 (1, 0)
h. Laboratory abbreviations warrant clarification (i.e., instead of FBG, state fasting blood glucose)
1 (0, 1)
i. Notations used in CV risk profile are unclear (i.e., instead of “Med Order” state Medication Order)
1 (1, 0)
3. Clinical Data Search application
a. Difficulty navigating Clinical Data Search to find desired information
6 (3, 3)
b. Struggled to “log-out” of Clinical Data Search after use
6 (2, 4)
c. Uncertainty with what data are available within Clinical Data Search limited use
5 (4, 1)
d. Desired access to medication fill history across all pharmacies
5 (3, 2)
e. Anticipated access to updated/completed medication lists
3 (2, 1)
f. Wanted access to discontinued therapy notes more explicitly
2 (2, 0)
g. Expected access to additional information such as patient biometrics and insurance information
2 (1, 1)
h. Ambiguity with where data originates from hinders use
1 (0, 1)
i. Unaware of PDF enlargement functionality
1 (1, 0)
j. Laboratory notations utilized are unclear (i.e., instead of “n ” state normal)
1 (1, 0)
Abbreviations: HCPs, health care professionals, including pharmacists and pharmacy technicians; HIE, health information exchange; IHIE, Indiana Health Information Exchange; PDF, portable document format; RPh, pharmacist; tech, pharmacy technician.
a Indicates a high impact finding from RUE to ideally fix prior to actual use.
Table 3
System usability scale[34 ] scores by health care professional type across all community pharmacy sites (n = 3)
System usability scale score
HCP type
n
M (SD)
95% Confidence intervals for the mean
Bangor adjective rating[27 ]
Pharmacist
8
74.1 (11.2)
(64.7, 83.4)
Good
Pharmacy technician
8
65.3 (16.2)
(51.8, 78.8)
Good
All HCPs
16
69.7 (14.2)
(62.1, 77.2)
Good
Abbreviations: HCPs, health care professionals; SD, standard deviation.
Note: System usability scale scores range from 0 to 100. The Bangor adjective rating mean SUS score for worst imaginable is 12.5, awful is 20.3, poor is 35.7, OK is 50.9, good is 71.4, excellent is 85.5, and best imaginable is 90.9.
Table 4
Tool utilization, task success, and average time on task rates by HCP per scenario (n = 16)
Pharmacist (n = 8)
Pharmacy technician (n = 8)
Scenario description
% accessed tool[a ]
% Task success
Average time on task (min:sec)
% Accessed tool[a ]
% Task success
Average time on task (min:sec)
1. New patient: expected HCPs to access progress notes, laboratory results and/or look for other medication history
Clinical data search: 100%
CV risk profile: 37.5%
75%
4:39
Clinical data search: 62.5%
CV risk profile: 37.5%
37.5%
4:25
2. Routine mediation synchronization order: expected HCPs to access progress notes, laboratory results and/or look for other medication history
Clinical data search: 100%
CV risk profile: 37.5%
75%
3:10
Clinical data search: 37.5%
CV risk profile: 12.5%
12.5%
2:15
3. Medication therapy management: expected HCPs to access progress notes, laboratory results and/or look for other medication history
Clinical data search: 75%
CV risk profile: 62.5%
87.5%
2:26
Clinical data search: 62.5%
CV risk profile: 12.5%
50%
2:40
4. Immunization history request: expected HCPs to access vaccine history
Clinical data Search: 37.5%
CV risk profile: 75%
75%
2:41
Clinical data search: 50%
CV risk profile: 62.5%
50%
1:45
Abbreviations: CV, cardiovascular; HCP, health care professional.
a Not mutually exclusive; an individual participant may have accessed both HIE tools or no HIE tools.
Note: Percent accessed tool defined by how many HCPs accessed each tool function to find desired information.
Note: Percent task success defined by how many HCPs used tools to access expected information for each scenario.
Note: Time on task was determined by reviewing timestamps on each video from the start of each scenario to the end. Start times were defined as the moment a participant moved their mouse after starting each scenario. End times were defined as when the participant verbally stated or confirmed they had completed the scenario OR stopped moving their mouse, whichever occurred first.
Discussion
This study is the first formative usability evaluation of an HIE interface prototype developed specifically with and for use by community pharmacy professionals. Pharmacists have a unique workflow and do not commonly utilize EHR systems utilized in health/hospital systems. Our scenario-based evaluation applied human factors methods to assess the usability of the HIE-Pioneer mock-up within pharmacy teams' existing software system workflow. Providing community pharmacies with desired, external patient records in an efficient and accessible manner through HIE is of utmost importance to ensure patient care is safe and efficient.[21 ] Our results indicate three key points: (1) the HIE-Pioneer mock-up supports learnability and efficiency for pharmacy professionals; (2) we identified key design aspects to improve HIE-Pioneer mock-up usability; and (3) pharmacy professionals, overall, viewed the HIE-Pioneer mock-up positively, with good ratings for satisfaction. Each of these points is further discussed in the following paragraphs.
The usability of the HIE-Pioneer mock-up generally fostered pharmacy professionals ease of learning (i.e., learnability) and efficiency. Overall, they were able to learn how to use the HIE-Pioneer mock-up on their own, with the time spent on each scenario generally decreasing throughout the usability session. Pharmacists and pharmacy technicians spent more time on scenario no. 1 (4:39 and 4:25 minutes, respectively) compared with the final scenario, scenario no. 4 (2:41 and 1:45 minutes, respectively), suggesting that the HIE-Pioneer mock-up became easier to navigate for pharmacy professionals throughout use. This is important since less time spent learning to use HIE could facilitate workflow and aid software implementation in a traditionally busy community pharmacy setting. Another study, Hohmeier et al, found that pharmacists who were accessing HIE data in community pharmacies to complete medication reconciliation spent an average of 21 minutes on each patient.[6 ] In comparison, scenario #3 in our study was a medication therapy management case (similar to medication reconciliation); pharmacists in our study spent an average of 2 minutes and 26 seconds on this particular scenario reviewing one medication. Importantly, pharmacists in Hohmeier et al's work accessed HIE through an external system login, rather than the integrated HIE interface we presented in this study, as well as reviewed an average of 15 medications per patient with real data compared with what was available for review in our formative evaluation of the HIE-Pioneer mock-up. Additional research utilizing integrated HIE with real patient data is needed. In our study, however, the minimal time that pharmacy professionals spent accessing needed HIE data to complete scenarios provides further evidence of “good” usability of our integrated HIE-Pioneer design.
Within the HIE-Pioneer mock-up, two applications were available—the Clinical Data Search application and the CV Risk Profile—and our results point to opportunities to improve the interface design for each application. Both pharmacists and pharmacy technicians tended to access the Clinical Data Search more frequently than the CV Risk Profile in most scenarios, except scenario no. 4. When asked why, most pharmacy professionals explained that they knew what to expect from Clinical Data Search compared with the perceived ambiguity of the CV Risk Profile application name. For CV Risk Profile, the application name, used for the associated access link ([Fig. 2 ]), was the most common usability problem and efforts are underway to modify this name for future HIE implementation.
Other usability problems that we identified, such as suboptimal placement of link-outs in the existing system and desiring increased accessibility (e.g., less clicks within workflow), were consistent with findings from a previous study on the original HealthDart application.[13 ]
[19 ]
[22 ] Participants also wanted to optimize existing features, such as the use of “tab” functions they were already used to using to increase accessibility of tools within their existing workflow. In our advisory panel sessions, suggestions for an alternative placement of link outs to the HIE data were discussed. Pharmacy professionals expressed a desire for HIE data to be accessed across multiple screens within dispensing software workflow and also emphasized the importance of minimizing the number of clicks needed to access HIE data. Both of these suggestions point to the need for HIE data to be readily accessible and highly efficient for optimal use in community pharmacies.
Usability satisfaction scores differed between pharmacists and technicians, with pharmacists having higher scores. Differences in perceptions in usability among participants based on role have been documented in previous literature, supporting that participants with different roles may have different needs from the system.[42 ] These scores also correlated with task success rates and application access rates—with pharmacists accessing HIE applications more often and completing more tasks successfully. These findings might be due to some technicians struggling to identify their role utilizing HIE, with one technician specifically noting “I am not sure what my pharmacist wants me to look for.” The ambiguity some technicians perceived with HIE and their role might contribute to the lower satisfaction (i.e., SUS) ratings by pharmacy technicians compared with pharmacists ([Table 3 ]). Even so, most pharmacy technicians were able to identify ways they could use HIE, with one technician stating, “[this mock-up] would be really useful.”
Involving both pharmacists and pharmacy technicians in the formative evaluation of the HIE-Pioneer mock-up was instrumental for planning the next steps in our iterative HIE design and implementation process.[22 ]
[43 ]
[44 ] Designing HIE in a way that is consistent with existing pharmacy workflows and pharmacy professionals' expectations should help promote HIE adoption, satisfaction, and sustainability.[18 ]
[19 ] Through our research, we identified key findings that will be addressed prior to full-scale HIE implementation. For example, a key change we are making is to increase the accessibility of HIE by reducing the number of required clicks to access HIE data by optimizing link-out placement for HIE access within the existing pharmacy software system, PioneerRx.
Although our study is novel, we recognize that there are some limitations. Think aloud technique is a known confounder for time measurement[45 ] and can artificially inflate time measurements and reduce efficiency; therefore, our time-on-task measurements provided here are likely an overestimate of the time it will take pharmacy professionals to utilize the HIE interface in clinical practice.[46 ] Although we had input from IT developers from all software vendors, conducted pilot testing with pharmacists, and chose areas of interface workflow we anticipated would be utilized frequently by pharmacy teams, pharmacy professionals sometimes expected to see different screens or functions that were not available within our prototype. Thus, through our study, we identified important clinical workflows that need to be supported by HIE software, highlighting the value of conducting usability testing prior to actual software implementation. One lesson learned included a need for multiple access points to HIE applications within the HIE-Pioneer mock-up design to support a wider variety of pharmacy workflows. The mock-up was designed initially with three access points, and after usability testing and advisory panel feedback, the future version will have upward of five access points to better support community pharmacy workflow. Pharmacists and pharmacy technicians have different workflows within the same system, and both offered unique perspectives and suggestions for access points pertinent to their roles. Multiple access points also reduce clicks needed for pharmacy teams to find pertinent HIE within their existing workflows. A second lesson learned was that naming HIE applications in an intuitive way for pharmacy team members was critical for promoting use. Both pharmacists and pharmacy technicians demonstrated reluctance to access “CV Risk Profile” because the HIE label name was ambiguous. In contrast, “Clinical Data Search” was more straightforward. The “CV Risk Profile” will be renamed to “CV Dashboard” in the future HIE-Pioneer interface as a result.
Therefore, compared with our HIE-Pioneer mock-up, some aspects of pharmacists and pharmacy technicians' perceived satisfaction with HIE interface usability are likely to improve once a fully functional HIE interface is implemented. Participants only interacted with the HIE-Pioneer mock-up on one occasion, limiting the ability to study recurring usability problems with the HIE interface design. However, we also collected feedback afterward from advisory panel sessions that will be utilized to inform a future version of the HIE-Pioneer interface. Additionally, this study had a limited, purposive sample size of three smaller community pharmacies and their employees. These pharmacies are highly engaged in community pharmacy advancement initiatives and may be more interested in access to HIE compared with other community pharmacy teams. However, our overall sample size of 16 participants aligns with sample sizes from published studies that evaluated the usability of other health care applications.[47 ] Therefore, additional usability problems might arise with a larger sample of pharmacy professionals. Sample size research by Faulkner and Nielson, however, indicates that our sample size of 16 participants is appropriate to uncover the most substantial usability problems.[30 ]
[33 ]
[48 ] Future research with the next iteration of an HIE interface for use by community pharmacy teams should include a larger sample for summative usability evaluation and a wider variety of pharmacist and pharmacy technician users.
Conclusion
This is the first evaluation of an HIE interface designed with community pharmacy professionals' input that enables pharmacy professionals to access statewide HIE data from within their existing dispensing software system. Incorporating pharmacist and pharmacy technician feedback through scenario-based usability testing and then through a follow-up advisory panel was instrumental to identifying ways to improve the HIE interface design prior to full-scale implementation. This research identified some components of HIE interface design that can be further improved for use by pharmacy professionals. Key results from our study include the identification of access points within the existing workflow for HIE access and ways to reduce clicks needed to access the desired information. Our study results can be applied to the design of future HIE systems designed for use by multiple different kinds of pharmacy professionals in outpatient settings such as independently owned or chain-based community pharmacies.
Clinical Relevance Statement
Clinical Relevance Statement
Community pharmacist and pharmacy technician access to HIE can lead to improved communication amongst HCPs involved in patient care and equip pharmacy professionals with needed information for improved clinical decision-making. This research provides a foundation for the development of sustainable and scalable HIE participation by community pharmacies.
Multiple-Choice Questions
Multiple-Choice Questions
Which of the following is a favorable design consideration for health information exchange access for health care professionals?
Multiple clicks to access information
Integration with existing systems
Access through only external platform login only
Incomplete patient data
Correct Answer: The correct answer is option b. By integrating health information exchange access into existing systems, “clicks” needed to access the information are reduced (eliminating answer A) and health care professionals do not need to log into an external platform (eliminating answer C). Having as much complete patient data as possible would be favorable for health care professionals to know what to expect when accessing the health information exchange (eliminating answer D).
Which of the following data elements are commonly available through health information exchange?
Medication fill history
Insurance information
Laboratory data
Patient phone number
Correct Answer: The correct answer is option c. Data elements, such as Insurance information and patient phone numbers, are not routinely collected as discrete elements in health information exchange (eliminating answers B and D). This information may potentially be included in progress notes. Additionally, medication fill history data are generally not captured by health information exchanges. These data are typically only available at the pharmacy, a patient fills their medications at and/or through the insurance company records (if applicable; eliminating answer A).
