Keywords
clinical informatics - professional training - education and physician - general health care providers - training and education requirements - education and strategies for health IT training - education and certification
Background and Significance
In the past 15 years, the use of the electronic health record (EHR) has been widely adopted by the U.S. medical system and is now the norm in academic centers. Accordingly, medical educators must equip trainees with skills necessary to navigate the EHR efficiently and effectively.[1] While recommendations underscore the importance of medical students having meaningful access to the EHR, its use presents unique challenges in undergraduate medical education.[2] Systems struggle to find ways to allow students to have access necessary for training while protecting patients from potential harm. Health systems are generally risk averse, and due to a desire to protect patients and concerns about payer restrictions policies are created that limit medical students' roles within the EHR.
Students' access to the EHR ranges from a strictly observational role to documenting a parallel word-processing software, to being able to write orders to be released by a senior provider, and to document with cosignature.[3] Possibly in part as a result of limited student access, challenges cited by residency programs indicate that we have not found a way to adequately train medical students on the use of the EHR.[4]
[5] The central importance of this training is highlighted in two of the Association of American Medical College's Core Entrustable Professional Activities (EPA) for Entering Residency: EPA 4, enter and discuss orders and prescriptions, and EPA 5, document a clinical encounter in a patient record.[6]
Traditionally, student access to an EHR—no matter how limited—requires site-specific onboarding through in-person classes or electronic modules presented via learning management systems.[7] This onboarding rarely, if ever, provides certification that is portable across health systems using the same EHR, and students are often granted access that varies across the systems through which they rotate within medical school, with less than half being able to document in patient charts in all the hospitals in which they train.[8] We found no examples of systems that allow for portability of onboarding certification across institutions nor any that allow for competency-based assessment for students to demonstrate that they have skills necessary to avoid duplicative onboarding. The need for site-specific credentialing for EHR access can be time consuming and often redundant, occupying students' time that could be better spent in clinical training. While site-specific differences in EHR implementation likely create a need for unique components to onboarding, we sought to centralize the majority of onboarding with a single curriculum that could be accepted within multiple systems.
The University of Minnesota is one of the largest medical schools in the country with over 900 medical students who rotate through clinical experiences across several medical systems in the state. Most of these systems use Epic, and each site requires its own onboarding. We wished to determine the expectations and potential cumulative time burden of Epic onboarding for each of the sites where our students rotate. We then aimed to determine the feasibility of collaborating to form a single certification and competency-based assessment for medical students accepted by all teaching health systems in the state that use Epic.
Methods
We used Kern's Six Step Approach to Curriculum Development[9] as a framework to create our curriculum and assessment, entitled “Epic101.”
-
Step 1–Identify problem/needs assessment. We surveyed the lead Epic educators responsible for training at the six health systems that use Epic and host medical students to understand required onboarding content, delivery method, and the expected time burden for learners. We also asked their willingness to join a collaborative effort to determine essential components for a single certification course. Based on the results of this survey, two authors (M.P. and M.S.) from the medical school and its affiliated health system planned to create modules and a competency-based assessment for a common certification course, vet them with the Epic educators, and oversee their implementation.
-
Step 2–Determine goals and objectives for the curriculum. We solicited topics that each group deemed important to include in this orientation and then refined this list with multiple iterations of a modified Delphi process via teleconferences and surveys. We used a 4-point scale (1–not necessary, 2–probably not needed, 3–probably needed, 4–essential) to assign relative importance to topics on our resulting list. Topics averaging a 3 or greater were included in the curriculum and are depicted in [Table 1].
-
Step 3–Identify an educational strategy. We chose online modules for several reasons: (1) this system was frequently used as part or all of the Epic onboarding across the systems, making buy-in and ability to collaboratively develop the curriculum more feasible; (2) online modules could be housed behind a fire-wall requiring student logins which complies with Epic policies on use of proprietary screenshots from the EHR; (3) the online modules required a minimum of ongoing faculty and staff effort; (4) students were able to complete the curriculum asynchronously, thereby not interfering with their clinical experiences; and (5) online modules allow for an environment mimicking the EHR where learners can click and navigate as they would in the patient setting. We created four online modules that addressed the agreed upon content areas: Epic environment overview, order entry, patient information review, and chart documentation. We used the Adobe Captivate software to create the modules because the lead developer had experience with this platform. Lead educators from the six institutions vetted and refined the modules over a period of 4 months. Representative screenshots from these modules and full learning objectives are provided in [Supplementary Material A] (available in the online version).
-
Step 4–Develop an assessment. We aimed to create an assessment that would allow for demonstration of competency within Epic and that would ultimately grant portable certification across the participating institutions. We used the EpicPlay environment, a parallel, non-live EHR that simulates the Epic system and allows for the creation of simulated patients. This approach allowed us to create case-based questions mirroring real world encounters and to embed answers within simulated patient charts. Unlike module-based EHR assessments that require a certain pattern of clicking to demonstrate skill attainment, our approach allowed for multiple paths to each answer, mirroring the multiple real world paths to find such information within the EHR. We defined the ability to correctly answer 80% of twenty questions as demonstrating competency within Epic. An example of a series of questions is provided in the [Fig. 1]. We provide the full assessment and key as [Supplementary Material B] (available in the online version).
-
Step 5–Implementation. We placed the modules and assessment in the students' online learning management system in April 2016. In May 2016, all rising third year medical students preparing for their foundational clinical clerkships were required to complete both the modules and the assessment. All rising fourth year students who had already used Epic at least once had the option to take the assessment only, although they also had access to the modules.
-
Step 6–Evaluation and feedback. Upon completion of the modules, students completed an anonymous survey that captured feedback on prior Epic experience, time requirement for module completion, level of comfort in using Epic upon entry into their first clinical rotation, and suggested future module topics. We also obtained feedback from all site Epic educators through regular email and phone communication. As a course-related activity with data collected as part of the routine class activity for program evaluation, the University of Minnesota Institutional Review Board considers the project exempt from review.
Table 1
Agreed upon essential skills for Epic101 curriculum
|
Skills
|
Mean score of importance for inclusion (1–4 scale)
|
|
Writing notes from scratch
|
4.0
|
|
Using chart review
|
4.0
|
|
Navigating results review
|
3.9
|
|
Writing notes using SmartPhrases
|
3.9
|
|
Finding your patient
|
3.9
|
|
Reviewing the problem list
|
3.6
|
|
Entering orders
|
3.1
|
|
Using order sets
|
3.1
|
|
Performing medication reconciliation
|
3.1
|
|
Customizing a patient list
|
3.1
|
|
Writing notes in NoteWriter
|
3.0
|
|
Using admission/discharge navigator
|
3.0
|
Fig. 1 Sample assessment questions requiring students to navigate simulated patients' chart.
Results
Prior to implementation of the Epic101 curriculum, all six institutions had required their own EHR training regardless of any previous training a student might have completed. Four of the six had an online component to the training, and three had an in-person requirement with trainers at the elbow of the students. The time requirement ranged from less than 2 hours (one site) to more than 4 hours (two sites), with the majority requiring between 2 and 4 hours. With the training required at each site, sometimes for each rotation, an individual student could be required to spend up to 25 hours in duplicative onboarding, depending on his or her rotation schedule.
We obtained buy-in from all six institutions that if we collaborated to create a curriculum and assessment that addressed the overlapping expectations for each institution, each would allow this single onboarding to serve as the credentials for their respective site, eliminating or decreasing the amount of onsite training required. Representatives from each of the sites reviewed and provided feedback on content of each module. This was repeated until we had agreement that the module content adequately covered the skills listed in [Table 1]. Examples of topics which did not reach the threshold for inclusion included InBasket Management and use of CareEverywhere to find patients' Epic records outside of the system. The workgroup discussed their inclusion and agreed they were not needed as core concepts for the medical student onboarding. Leadership then reviewed the drafted modules to ensure that it met their expectations before going live with the curriculum.
In the pilot year of the curriculum, 443 students (94%) successfully completed the Epic101 assessment. All but three students did so on their first attempt, and of these, 438 students (99%) completed the modules and assessment in less than 2 hours. A total of 173 students completed the postassessment survey (response rate 39%), and of those, 125 (72%) felt comfortable in their ability to use Epic effectively on their next clerkship.
As a result of the pilot year, five of the six sites have eliminated site-specific Epic training for students. One site still requires a half-hour onsite session for site-specific skills for inpatient clinical rotations. This change resulted in as much as a 20-fold reduction in time spent in Epic onboarding for students during their clerkships, allowing for more time in the clinical setting. After one year's experience, all six sites plan to use the curriculum and honor the completion certificate moving forward.
Conclusion
In our creation and pilot of the Epic101 curriculum, we demonstrated that multiple different clinical partners could overcome the challenges of institutional bureaucracies to create a portable curriculum that has been widely adopted, has saved time for both Epic trainers at clinical sites and students alike, and has provided students comfort in their readiness to use the EHR during their clinical rotations. Furthermore, our curriculum focuses on specific medical student tasks identified by the training institutions, in distinction to a broader array of skills addressed in a self-paced user guide an EHR software may incorporate within its platform.
This first cycle demonstrated the curriculum's feasibility with a 99% completion rate. It demonstrates a novel use of a preclinical competency-based assessment completed in the EHR environment in which the students will work. While module-based EHR assessments typically require students to follow an exact clicking algorithm to answer the question correctly, our assessment more closely resembles clinical practice in that it acknowledges that multiple roads lead to the same result within the EHR. This approach to assessment allows students the opportunity to demonstrate that they know how to find a laboratory result, for example, without being prescriptive in how they find it. While others have created vendor-agnostic student EHRs for training,[10] the ability to leverage the EpicPlay environment, part of the out-of-the-box Epic platform, allows for a logical testing environment which mimics their eventual workspace. Additionally, we were able to create questions that went beyond simply data retrieval, or skills that could be accomplished by simply using the chart search feature. For example, we were able to assess the student's ability to successfully enter an order by asking details about what is shown when the order is pulled up (e.g., the Laboratory Code). If the student is able to successfully find this detail, it means he or she was able to find the order successfully.
We encountered several stakeholders along the way who felt it was unlikely we would be able to get six distinct systems to agree to a single onboarding process. Accordingly, our most significant outcome of this pilot project may be the fact that such a collaboration was, in fact, possible in the first place. We feel there are a few key factors that led to buy-in across institutions. First, we identified a common problem in the duplicative onboarding and offered an opportunity to collaborate for a solution. Second, we proposed that we (as the University site) would take the lead on creating the infrastructure of the modules, assessment, and tracking. Offering to take the burden of the logistics of making the curriculum, while keeping the equal stakes in input, may have contributed to the successful buy in. Additionally, we agreed to move forward with an “all for one, one for all” approach where we agreed that we would repeat feedback cycles until we were all in agreement that the modules met our needs. This so-called “Muska-Peer Pressure” helped move the project along as all the stakeholders were given equal content and veto power, yielding a true collaborative effort.
There are limitations in our work. First, the project was not designed as a controlled study, so while the majority of students (72%) indicated they felt prepared to use Epic on day 1, we are unable to compare their confidence with that of students who entered clinical rotations without this training. Second, survey questions were self-reported, so the comfort level may not reflect the skill level in EHR navigation. Third, as it is meant to replace the redundant onboarding addressing the logistics of Epic training, this curriculum focuses merely on the nuts and bolts of Epic use, not on the nuances of EHR such as professionalism, ability to maintain rapport with patients, etc.[1]
[3] We do, however, address these important topics elsewhere in our medical school curriculum including opportunities to be evaluated on EHR use in simulated patient encounters as part of a more comprehensive curriculum addressing the provider's role in the EHR ecosphere. Fourth, the EpicPlay environment does not exactly mimic real Epic charts that have more content and granularity. However, it does provide an environment that is sufficiently challenging for novice users. Finally, we were not able to address certain aspects of training such as validating the correctness of individual student login and set-up for each health system. This is still a task that needs to be addressed by training teams at each health system.
This collaborative model serves as a proof of concept and is easily replicable by medical schools at which students rotate across multiple health systems. While use of this model necessitates a single type of EHR across multiple training sites, the consolidation of EHR vendors makes this situation likely around the country, and it would be possible to repeat this process for different EHR vendor cohorts within a training system. Our goal is that other programs with multisystem training sites with a single type of EHR will consider this approach to minimize training burden for their learners, thus allowing them more time in the clinical setting. The assessment relying on answers embedded in the charts of simulated patients within the EpicPlay environment requires minimal work to create and can likely be mirrored in other training programs. All six sites wish to continue this universal EHR certification and assessment, and we aim to continue this program for our residents as well as for our students, and to systematically gather feedback about comfort/EHR skill level during clinical experiences.
