Keywords
Electronic health records and systems - Knowledge translation - Internet and the Web
technology - Strategies for health IT training - Employee orientation program
1. Background and Significance
1. Background and Significance
The promise of electronic medical records (EMRs) to improve quality, productivity,
and efficiency in health care organizations, as well as incentives and penalties for
use and nonuse of such systems, has led more and more organizations to implement EMRs
[[1], [2]]. However, this implementation has been challenging [[3], [4]], and subject to both individual and organization-level barriers for use [[5]–[9]]. Research has consistently demonstrated that training on health information technology
can positively influence a provider’s willingness and ability to use EMRs [[10], [11]]. However, how to best train busy clinicians to incorporate and efficiently use
an EMR remains an outstanding question.
A variety of factors can influence EMR use, including the timing of training, how
well training is targeting user needs, and accessibility of on-site support [[12]–[14]]. A growing body of research examines the benefits of different training approaches
[[13], [15]–[17]], but most of the described methods are instructor-led. Instructor-led training
(ILT) typically involves a computer lab and limited session dates. ILT can be expensive
and time-consuming [[18]–[20]], and this ‘one size fits all’ approach falls short for some end users.
The eLearning method of EMR training offers the potential for long-term cost savings
as compared to ILT [[19]]. eLearning’ refers to user-directed online education, or the use of technology
to deliver learning solutions with high levels of interactivity, flexibility, and
communication [[21]]. Evidence demonstrating the effectiveness of eLearning is growing [[22]], yet the experience of converting EMR training from an instructor-led to an eLearning
approach has not been described in the literature. We examined the impact and return
on investment of conversion to an eLearning format at our large academic medical center
(AMC).
2. Objectives
The system-wide implementation of the current EMR at The Ohio State University Wexner
Medical Center (OSUWMC) occurred in 2011 [[23]], led by the organization’s Information Technology (IT) department. OSUWMC’s IT
department includes a Training and Optimization team established to provide training
for new end users, assist current staff in optimizing their EMR use, and ensure that
end users remain current on evolving functionality. After the Go-Live event, this
Training and Optimization team redesigned and streamlined the organization’s EMR education
approach.
Under the revised ILT curriculum, providers new to OSUWMC experienced “layered” EMR
training. This curriculum started with essential content and continued with additional
training based on the user’s specialty. These ILT sessions were offered twice a month.
We were onboarding an increasing number of physicians and residents with prior EMR
experience. Many of these physicians were already familiar with the system in use
at OSUWMC when they arrived for training. Therefore, we were challenged to modify
our training to meet needs based on varying levels of prior system experience.
eLearning introduces the ability to meet learners where they are by delivering content
dependent on learners’ demonstrated EMR competency. We hypothesized that a transition
from ILT to eLearning would increase user satisfaction and reduce training time, freeing
more time for revenue-producing patient care.
3. Methods
This case study was conducted at The Ohio State University Wexner Medical Center (OSUWMC),
an AMC comprised of six hospitals, two campuses, and 46 outpatient sites in Columbus,
Ohio. Close to 24,000 staff members utilize the EMR system, including approximately
1,100 attending physicians, 800 residents and 600 advanced practice professionals
(APPs). All of these physicians and APPs recently transitioned from ILT to eLearning
for the training required prior to obtaining access to OSUWMC’s electronic medical
record.
We designed our ILT using a competency-based approach, requiring learners to demonstrate
knowledge about and skill in using the EMR. This foundation created an easier transition
from ILT to self-paced eLearning. We also designed workflow and scenarios into our
ILT to enhance transfer of training to patient care. In the transition to eLearning,
we identified a set of eLearning Development Principles (►[Table 1]) to ensure an experience that flows and engages learners.
Table 1
Development Principles for eLearning Success
|
Guiding eLearning Principles
|
|
Make sure there is consistent presentation of material
|
|
Use a project management system for tracking development
|
|
Have leadership and operational support
|
|
Utilize a standard development process
|
|
Enforce stringent quality assurance reviews
|
|
Ensure staff have time, skills and software resources to be successful
|
|
Use a robust Learning Management System
|
|
Make smart choices about investments
|
|
Adhere to design requirements:
-
Standard style guides and templates
-
No audio due to muted computers in patient care areas and sound maintenance considerations
-
Short and digestible content, with no more than 30 minutes per lesson
-
Each course typically includes multiple lessons including guided and un-guided interactions
|
Development of eLearning requires specialized skills that are different from those
needed for ILT development. We did not hire any new team members. Instead, we identified
new roles and dedicated staff time to developing expertise in these roles (►[Table 2]).
Table 2
Roles, Duties and Skill Sets Required for eLearning Oversight and Production
|
Role (% of Total Development Time)
|
Primary Duties
|
Skill Set Required
|
|
|
|
|
|
|
-
Project manages storyboard and Captivate production
-
Project manages quality review process
-
Instructional design review of storyboards
-
Maintains templates for storyboards, Captivate and Lectora
|
-
eLearning design
-
Instructional design
-
Captivate development
-
Lectora development
|
|
|
-
Owns content
-
Determines design approach for curriculum
-
Completes content and quality review of storyboards, interactive demos, final courses
-
Designs test questions, exercise assessments
|
|
|
|
-
Designs storyboards
-
Uses storyboard blueprint, screen shots and Captivate to produce interactive demos
-
Updates Captivate lessons per quality review until final version is accepted
-
Uses Lectora template to produce courses
|
-
Storyboarding
-
PowerPoint
-
Captivate
-
Lectora
|
|
|
-
Owns writing standards
-
Completes quality review of storyboards and Captivate files for standards adherence,
structure and consistency
-
May develop storyboard and Captivate files
|
|
Our eLearning curriculum design is modular. Each curriculum has a set of courses and
each course has a set of lessons, making it easy for first time learning and subsequent
review of specific topics. Courses are taken in workflow order. Each lesson is designed
to teach EMR functionality and application in a typical scenario for a role (i.e.
ambulatory or inpatient clinician). Lessons allow learners to interact with the content
as if they are in the live EMR. Knowledge assessments are integrated into each course
to ensure that progress within a curriculum is based on demonstrated knowledge. Additionally,
learners are required to demonstrate their knowledge in a competency assessment during
a 1:1 onboarding session with a trainer. The competency assessment process is the
same as for ILT.
It was essential to create a structured eLearning design/development process that
allowed for accurate conversion of a large number of classes in an efficient, economical
manner. Our process included five steps.
Step 1: Analysis
In the first phase of each class conversion, we restructured ILT content to align
with our eLearning Development Principles (►[Table 1]). The deliverable was a design approach document that mapped classroom topics to
eLearning courses and lessons. Most classes mapped to between three and five eLearning
courses.
Step 2: Design and Storyboarding
The eLearning Developer worked in close collaboration with the Principal Trainer,
or content owner, to convert classroom content to visual storyboards for each lesson.
A storyboard template provided a standard construct for developers and trainers to
use in the design of each lesson while allowing for creativity in the presentation
of content. Using the storyboard template for each lesson, the eLearning developer
proceeded through six activities:
-
Follow a classroom script in the EMR to take screen shots;
-
Insert the screen shots in a Microsoft PowerPoint document;
-
Lay out the text for each slide, including important notes and callouts;
-
Look for opportunities to introduce learner engagement activities;
-
Work with the Principal Trainer to finalize each storyboard; and
-
Save final screen shots for production in Adobe Captivate.
When a storyboard draft was complete, the Principal Trainer, Project Manager, Instructional
Design Lead and Technical Editor reviewed the draft for quality. They evaluated the
content and ensured adherence to design principles and eLearning standards.
Step 3: Interactive Lesson Development
Once the storyboard was approved, eLearning developers converted it to Adobe Captivate,
where the lessons came to life. Developers used a Captivate template that set design
standards for all interactive demos.
When the Developer completed a Captivate file, the Principal Trainer, Instructional
Designer and Technical Editor conducted a review to confirm that all content was transferred
accurately from the storyboard, and the interactive demos were operating correctly.
Step 4: Course Development
When all of the interactive demos were complete, the Instructional Designer used Lectora
to publish each course for use in a Learning Management System (LMS). The Lectora
portion of each course includes information about audience and objectives, a menu
of interactive demo lessons, and assessment.
As part of course development, the Principal Trainer designed assessment questions,
including both multiple choice and scenario-based questions that test learners on
applying what has been learned. Our assessments require learners to answer 80 percent
of questions correctly to pass. If learners do not achieve 80 percent, they can review
the course and attempt the assessment again until they pass. A course is ready for
deployment after a final quality assurance checks by a subject matter expert, instructional
designer, technical editor, and an expert in the EMR software system in use (Epic™).
Step 5: Deployment
New clinicians were enrolled in the curriculum via the LMS. For the first several
months after deployment, we had clinicians complete eLearning in the classroom to
make sure the courses worked as intended. The Principal Trainer facilitated these
sessions to troubleshoot any issues that emerged. Once we were confident in the eLearning
curriculum, we developed instructions so that clinicians could complete courses from
any location, any time, using the password-protected LMS. Clinicians are able to call
the Medical Center help desk with questions. Technical eLearning/LMS issues are addressed
by the first line help desk staff, and content issues are routed to an on-call trainer.
During the initial rollout, several issues emerged related to coordination with approximately
100 staff across the medical center with responsibility for onboarding providers.
We had to assure that these staff knew who to contact at the Training Center to have
modules assigned to new providers, and that the Training Center staff had a detailed
decision tree in place to accurately assign the correct training to providers based
on their specialty and practice locations. Onboarding staff were also educated on
the need to schedule time in the providers’ first days of clinic for 1:1 onboarding
sessions and competency assessment.
4. Results
All new providers now use eLearning instead of ILT for EMR training. The new structure
for basic provider training consists of four curricula: Basic, Standard, Specialty,
and Personalized Content. The online structure allows users with previous EMR system
experience to test out of Basic Content, saving physician time and organizational
resources.
The cost of converting a course to eLearning quickly adds up, in our case, quickly
exceeding an investment of $40,000 to $50,000 for the primary provider eLearning curriculum.
However, the potential for savings comes from reduced non-productive time that clinicians
spend in classroom training [[19]].
To maximize our return on investment, we were selective about where to invest in conversion
of ILT to eLearning. Some classes continue to be taught in person due to specialized
content, or because the class impacted so few users that investment in development
time could not be recuperated. In deciding where to implement replace ILT with eLearning,
we considered the volume of staff trained, the hourly rate of affected staff, and
EMR workflow stability.
We train approximately 400 high hourly rate providers (physicians and advanced practice
professionals) each year. Hours gained can be applied to revenue-producing activity,
making our provider classes good choices for eLearning conversion. Our first eLearning
courses showed a significant reduction in training time compared to the ILT format
(►[Table 3]). Because eLearning is self-paced and can meet providers where they are, providers
completed training via eLearning approximately 45% faster than classroom training.
Given the volume of new providers in our organization, these time savings can be easily
translated into additional patient care time and a significant revenue opportunity.
This conversion also enabled us to return time to our trainers for 1:1 end user optimization
work.
Table 3
Gains
|
Who
|
1 Year Gain (hours)
|
Available for
|
|
Faculty
|
700
|
Clinical Service (175,5 day clinics)
|
|
Residents
|
920
|
Clinical Service (230,5 day clinics)
|
|
Trainers
|
450
|
Support and Optimization
|
Our standard practice is to survey learners after ILT, and we continue to obtain feedback
after eLearning courses. In addition to the return on investment, it is important
that providers like the eLearning format. Learners were surveyed upon completion of
the eLearning on how we could improve the training experience. Responses were positive
overall, with providers reporting that eLearning was comprehensive and easy to navigate.
Providers with previous experience using our EMR found the self-paced format especially
beneficial because it allowed them to move more quickly or test out of training. A
few clinicians noted that they would like to be able to ask questions, which is not
directly possible within the LMS. We also noted that the hands-on practice exercises,
when not programmed as requirements for module completion, were sometimes being neglected.
Our developers are working to address these challenges in future eLearning updates.
5. Discussion
Successful conversion of our provider trainin from ILT to eLearning took considerable
planning and resources. The goal was to deliver a high-quality product that would
support our providers with practical EMR training in less time. The literature on
continuing education in the health professions has shown that online format can be
just as effective at achieving learning outcomes as classroom-based instruction [[24], [25]]. The flexibility of the online format is valued by busy clinicians [[26]–[28]]. We noted additional positive impacts on the organization as a result of this conversion,
including time savings.
Many new clinicians arrive at OSUWMC with experience using the same EMR at other agencies.
It is important to minimize onboarding time so clinicians can be productive as soon
as they are on site. Completing training remotely via eLearning, even before arriving
at our facility, allows clinicians to begin patient care earlier.
6. Conclusions
Converting provider training to eLearning impacted our organization in three major
areas. First, our providers now spend less time in class and can learn through personalized,
self-paced lessons. Learner satisfaction increased with this delivery method, which
has also created a positive image for our organization during provider onboarding.
Second, our Training and Optimization staff are out of the classroom and have time
to provide more on-site end user optimization. Finally, at the organizational level,
the return on our investmet in conversion was robust, reflected in both time and cost
savings. Clinicians spend less time in training and more time in clinical roles.
Our eLearning development process will continue to be refined. We are currently pursuing
the conversion to eLearning for other clinical roles. We plan to make our next conversion
decisions based on the criteria we have established for return on investment and adherence
to our eLearning Development Principles.
Multiple Choice Questions
Multiple Choice Questions
1. In changing from ILT to eLearning, what are the key factors for an organization
to include?
-
A structured development and QA process
-
Focus conversion effort on small specialty classes for maximum benefit
-
The need to cover all policies and procedures in the training
-
The importance of outsourcing the development process
Correct answer: A – While there are many benefits to eLearning, development of eLearning
modules can be expensive and time consuming. The cost of poor quality, in terms of
rework and/or incorrect training, is high and should be avoided. A structured development
and QA process can assure that eLearning modules are high quality and correct. 2.
Why would an organization consider switching to eLearning?
-
Assure that training is “one size fits all”
-
Assure that all learners complete training on site
-
Reduce training time due to self-paced nature
-
eLearning development is quick and inexpensive
Correct answer: C – Self-directed eLearning allows learners to complete training at
their own pace, allowing those who have had experience with this particular software
to go through the material more quickly, and those who need more time can complete
it at their own pace without feeling rushed.
