Keywords
medical student education - undergraduate ophthalmology education - severe acute respiratory
syndrome coronavirus 2 pandemic - coronavirus disease - COVID-19 - telemedicine -
telehealth - remote learning - online teaching
The severe acute respiratory syndrome coronavirus 2 pandemic began in Wuhan, China
at the end of 2019 and led to unprecedented changes in medicine.[1]
[2] As social distancing measures were implemented, medical schools were forced to consider
the implications coronavirus disease (COVID-19) would have on the structure of medical
education.[3] On March 17, the Association of American Medical Colleges recommended to pause all
medical student clinical activities.[4] This led to rising third-year medical student (MS3) and fourth-year medical student
(MS4) to turn to virtual learning, instead of hands-on education in the hospital.
Medical students interested in ophthalmology have been significantly impacted by these
changes due to the limited exposure students have to the field.[5]
[6]
[7] Ophthalmology is not unique among medical specialties that rely on physical provider–patient
interaction. However, common ophthalmic screening can be performed virtually. There
is an opportunity for students to be incorporated into telehealth initiatives. Coupled
with online learning, this model could teach clinical pearls and familiarize the student
with virtual health care.
Online learning models have shown to be successful in other fields of medicine.[8] In one study, the online platform Zoom has shown to be effective in team-based learning
for pathology, with a 96.5% attendance rate and an 85% satisfaction.[9] In another institution, video-conferencing software Google Hangouts is used to teach
clinical anatomy.[10] Surgical departments in the United States have been adapting remote curricula in
topics such as neurosurgery.[8] These efforts highlight the potential of online teaching during this unprecedented
time.
It is clear that the absence of medical students in the clinical environment has negatively
impacted medical education during COVID-19. Thus, there is a need to restructure ophthalmic
medical education. Here, we present a 4-week novel ophthalmology curriculum for clinical
medical students, which begins to explore the integration of students into telemedicine
platforms and provides both group and self-study sessions.
Methods
This study was approved by the Emory University Institutional Review Board (00000487).
Between April 18, 2020, and May 15, 2020, a 4-week teleophthalmology elective was
offered to Emory University School of Medicine rising MS3 and MS4 via the video-conference
software Zoom (San Jose, CA). The elective was 1 of 34 virtual electives offered during
COVID-19 to mitigate educational interruptions. Other electives were 1 to 4 weeks
in length and topics ranged from the COVID-19 pandemic and epidemiology to specialty-specific
courses in medicine and surgery. The course was codirected by the school's director
of medical student education (E.B.G.) and Grady Memorial Hospital's chief of ophthalmology
(Y.M.K.). The elective was proposed and organized by two rising MS4s (S.N.D. and O.E.U.).
Course Curriculum
The elective's mission was to promote continued learning in ophthalmology through
six different learning objectives (LOs) ([Table 1]). There were no course prerequisites required for enrollment. The elective was graded
on a pass–fail system. Course requirements included completion of self-study and chart
review assignments, a peer presentation, and attendance at all in-class activities
and didactics sessions.
Table 1
Teleophthalmology course learning objectives
|
1
|
To familiarize students with components of the history and physical specific to ophthalmology
|
|
2
|
To learn differences between televisits and in-person ambulatory ophthalmology visits
|
|
3
|
To identify “high-risk” ophthalmic conditions prioritized in televisits
|
|
4
|
To become acquainted with the management of high-risk ophthalmic conditions
|
|
5
|
To extract relevant data from medical records for teleophthalmology visits
|
|
6
|
To engage with ophthalmology faculty and/or patients through Zoom
|
The first component of the course was online self-directed learning, which consisted
of prerecorded lectures and interactive online activities ([Fig. 1]). The first week provided an introduction to the eye and covered topics on red eye
and acute vision loss. The second week focused on chronic vision loss, systemic disease,
glaucoma, and retinal diseases. The third week concentrated on orbital trauma and
oculoplastic surgery. The final week was devoted to pediatric ophthalmology, neuro-ophthalmology,
and ocular oncology. Students were required to submit proof of completion of interactive
online activities.
Fig. 1 Resources utilized for self-study activities.
The second component was student presentations held twice a week. Each student gave
a 15-minute case-based session to teach fellow students about an ophthalmic disease.
They were tasked to discuss if the case was “high risk” and should be seen emergently,
urgently, or remotely via telemedicine. Topics chosen by MS4 course organizers included
bacterial conjunctivitis, corneal ulcer, uveitis, age-related macular degeneration,
diabetic retinopathy, retinal detachments, retinal vascular occlusions, primary open
and angle closure glaucoma, ocular chemical burns, ptosis, lacrimal disease, nystagmus,
strabismus, diplopia, ischemic optic neuropathies, optic neuritis, and ocular melanoma.
Presentations were evaluated by the MS4 course organizers, with additional 5 minutes
of live feedback from peers.
The third component was case-based discussions led by department faculty. There were
eight discussions: Acute vision loss, red eye, chronic vision loss, oculoplastic surgery,
ocular trauma, neuro-ophthalmology, pediatric ophthalmology, and ocular oncology.
Each session was 1 hour. Students were required to attend each session with audio
and video settings turned on and participate in discussions. Faculty were provided
with open-access American Academy of Ophthalmology faculty discussion slides but were
permitted to use their own presentations.
The fourth component was optional telehealth observations at Emory Eye Center, offered
in the fourth week. All interested students were required to complete the Emory Healthcare
“Telehealth for Providers” online training. Each visit started with a video discussion
between the patient and ophthalmic technician, who asked if the patient would be comfortable
with a learner present. If so, the technician e-mailed the Zoom link of the virtual
room to the student and formed a three-screen Zoom conference. The student took the
history and performed a focused ophthalmic examination, as the technician recorded
the findings and conducted additional tests. Once the technician exited, the faculty
member virtually entered the room. The student presented the findings and repeated
parts of the examination requested. Each encounter was followed by a 5-minute feedback
session with the technician and faculty member.
The fifth component was chart review activities. All students with access to the electronic
medical record (EMR) platform engaged in remote chart review projects aimed at helping
the residents working at the Grady Eye Center during the COVID-19 outbreak. Through
review of medical records, students created lists of patients with “high-risk” conditions
who required prompt evaluation. Each student was assigned 15 patient charts per week
and assignments were reviewed by the MS4 course organizers for completion.
Student Assessment
An anonymous electronic postcourse survey was made on Google Forms (Mountain View,
CA) and distributed to students via e-mail ([Supplementary Table S1], available in the online version). Each survey had 20 questions, including 11 10-point
Likert scale questions. All students completed an electronic informed consent. Each
survey contained baseline demographic characteristics, such as perceived knowledge
level and interest in ophthalmology. Questions about LOs and free-response feedback
were incorporated. Students who participated in a televisit were asked to comment
on their experiences. An anonymous postcourse multiple-choice test was also distributed
([Supplementary Table S2], available in the online version). The test consisted of 15 multiple-choice questions
selected from the United States Medical Licensing Examination World (UWorld) Step
2 Clinical Knowledge (CK) question bank (Dallas, TX). This subscription-based board
review question bank is commonly used by clinical students nationwide.[11] Each question evaluated a topic covered by the curriculum. The answer and mean user
score for each question were obtained by the medical student course organizers, who
had personal access to the question bank.
Statistical Analysis
The survey and test data were visualized on Microsoft Excel (Microsoft, Seattle, WA).
Descriptive data analysis on student characteristics of interest was conducted. Preliminary
Shapiro–Wilk's tests revealed nonparametric distribution for all data groups and subgroups.
Thus, the Mann–Whitney U test was used for independent continuous data, Wilcoxon's signed-rank test for paired
continuous data, and Fisher's exact test for categorical data. All analyses were conducted
using XLSTAT for Microsoft Excel (Addinsoft, Paris, France), with a p-value of significance set at 0.05.
Results
A total of 18 students enrolled in the 4-week elective. All students completed the
course survey and test. Eleven students (61.1%) were women and seven were men. Eleven
(61.1%) students identified as white. There were 12 rising MS3s and 6 rising MS4s.
The mean (standard deviation [SD]) age was 24.8 (2.1) years. Most students were comfortable
using Zoom. Age and gender of students were not significantly different between the
groups. A higher number of MS4s planned to apply to ophthalmology residency and reported
higher baseline interest compared with MS3s, yet the difference did not meet statistical
significance for either end point. However, MS4s reported a significantly higher baseline
knowledge in the field than MS3s (p = 0.02). Baseline class characteristics are provided in [Table 2].
Table 2
Baseline characteristics of 18 students in the teleophthalmology course
|
Characteristics
|
Class (n = 18)
|
MS3 (n = 12, 66.7%)
|
MS4 (n = 6, 33.3%)
|
p-Value
|
|
Mean age, y (range)[a]
|
24.8 (2.1)
|
24.4 (2.4)
|
25.5 (1.0)
|
0.07
|
|
Female sex, n (%)[a]
|
11 (61.1)
|
7 (58.3)
|
4 (66.7)
|
1.0
|
|
Race—white, n (%)[a]
|
11 (61.1)
|
5 (41.2)
|
6 (100.0)
|
0.1
|
|
Planning to apply to ophthalmology residency, n (%)[a]
|
8 (44.4)
|
4 (33.3)
|
4 (66.7)
|
0.3
|
|
Perceived knowledge in ophthalmology (SD)[b]
[c]
|
5.3 (1.7)
|
4.6 (1.1)
|
6.7 (2.1)
|
0.02
|
|
Interest in ophthalmology (SD)[b]
[c]
|
5.8 (2.8)
|
4.9 (2.9)
|
7.5 (1.8)
|
0.07
|
|
Comfort level using Zoom teleconferencing (SD)[b]
[c]
|
9.2 (1.1)
|
9.0 (1.2)
|
9.5 (0.8)
|
0.4
|
Abbreviations: MS3, third-year medical student; MS4, fourth-year medical student;
SD, standard deviation.
a Comparison using Fisher's exact test between MS3 and MS4.
b Ten-point Likert scale, with 1 indicating low and 10 indicating high.
c Comparison using Mann–Whitney's U test between MS3 and MS4.
Fulfillment of Learning Objectives
The majority of students favorably reviewed the course and its fulfillment of LOs,
with a mean overall rating of 8.1 (range, 6.7–8.8). The ratings (SD) were 8.6 (1.3),
7.3 (1.5), 8.8 (1.2), 8.6 (1.4), 8.3 (1.6), and 6.7 (10) for LOs 1 to 6, respectively.
MS3s reported equal or higher ratings across all LOs compared with MS4s, with mean
overall ratings of 8.2 for MS3s and 7.7 for MS4s. A statistically larger number of
MS3s reported the course met LO5 “extracting data from electronic patient charts”
compared with MS4s, with ratings of 8.8 and 7.5 for MS3s and MS4s, respectively (p = 0.02) ([Supplementary Table S3], available in the online version).
Ophthalmology Knowledge and Interest
Overall, the mean (SD) baseline self-reported ophthalmic knowledge of the class was
5.3 (1.7), with ratings of 4.6 (1.1) by MS3s and 6.7 (2.1) by MS4s ([Table 3]). There was a significant increase in self-reported knowledge following course completion,
with a mean (SD) class rating of 8.1 (1.1) (p < 0.001). Students from both academic years reported significant enhancement in knowledge,
with final reported ratings of 8.1 (0.9) (p = 0.002) and 8.0 (1.4) (p = 0.04) for MS3s and MS4s, respectively.
Table 3
Baseline and postcourse comparison of ophthalmology interest and knowledge by student
academic year
|
Study group
|
Mean baseline knowledge (SD)
|
Mean postcourse knowledge (SD)
|
p-Value[a]
|
Mean baseline interest (SD)
|
Mean postcourse interest (SD)
|
p-Value[a]
|
|
Class (n = 18)
|
5.3 (1.7)
|
8.1 (1.1)
|
<0.001
|
5.8 (2.8)
|
8.1 (1.3)
|
0.003
|
|
MS3 (n = 12, 66.7%)
|
4.6 (1.1)
|
8.1 (0.9)
|
0.002
|
4.9 (2.9)
|
7.8 (1.3)
|
0.01
|
|
MS4 (n = 6, 33.3%)
|
6.7 (2.1)
|
8.0 (1.4)
|
0.04
|
7.5 (1.8)
|
8.7 (1.2)
|
0.1
|
Abbreviations: MS3, third-year medical student; MS4, fourth-year medical student;
SD, standard deviation.
a Comparison using Wilcoxon's signed-rank test.
Similar to baseline knowledge, the mean (SD) baseline interest of the class was rated
at 5.8 (2.8), with mean MS3 rating of 4.9 (2.9) and mean MS4 rating of 7.5 (1.8) ([Table 3]). A 2.3-point increase in ophthalmology interest was observed, with a final mean
(SD) class rating of 8.1 (1.3) (p = 0.003). Both student groups reported an increase in interest in the field, with
final mean (SD) ratings of 7.8 (1.3) and 8.7 (1.2) for MS3s and MS4s, respectively.
The 2.9-point increase in MS3 interest was significantly higher than baseline rating
(p = 0.01). However, the 1.2-point increase in MS4 interest did not achieve significance
(p = 0.1).
For the postcourse test, the mean class score (SD) was 94.8 (7.1), with a range of
77.8 to 100.0. The mean score for UWorld users for the 15-question test was 74.1 (12.9),
with a range of 50.0 to 94.0. Overall, the class performed significantly higher than
the average UWorld user (p < 0.001). This significance persisted when the class was grouped by academic year
([Fig. 2]). The class demonstrated higher scores than UWorld users for each topic. There was
no significant difference between MS3 (SD) and MS4 (SD) performance, which were 96.1
(5.3) and 92.2 (12.2), respectively (p = 0.35).
Fig. 2 Test results of the overall class, MS3s, MS4s, and UWorld users. Note: Error bars
reflect standard deviation. Comparison using Mann Whitney U-test. ∗∗∗
p < 0.001. MS3, third-year medical student; MS4, fourth-year medical student; UWorld,
United States Medical Licensing Examination World.
Telehealth Experience and Chart Review Activities
Eleven out of 18 (61.1%) students observed a telehealth experience. When the class
was stratified by telehealth observation, there were no significant differences in
baseline parameters and most LO ratings between observers and nonobservers. However,
LO5, “extracting relevant data from electronic patient charts” was rated significantly
higher by telehealth participants, with a mean (SD) score of 9.0 (1.4) compared with
7.9 (1.6) for nonparticipants (p = 0.03). Ten (90.9%) telehealth observers responded to the open-response questions.
Seven students highlighted positive interactions with faculty. Five students suggested
areas of improvement, including increasing the number of visits and providing students
with the patient charts prior to each visit.
Fifteen out of 18 (83.3%) students engaged with patient care virtually through participation
in the chart review activities. The weekly chart assignments aided residents and faculty
in triaging which patients needed to be seen urgently in clinic or via telehealth
visits. While helpful to the residents, these chart review activities received mixed
reviews from students. Two students enjoyed this component of the course, citing its
utility in offering an introduction to the EMR system prior to starting clinical rotations.
However, two students did not see educational benefit in the chart review activities,
and one additional student suggested coupling chart review activities with telehealth
visits for increased student engagement.
Qualitative Responses
A total of 17 (94.4%) students delivered open-response feedback. Fourteen students
praised the chosen self-study learning materials, seven students applauded the course
organization, five students liked case-based discussions, five students reported they
enjoyed peer presentations, and three students appreciated the flexibility of the
course schedule. There were several suggestions for improvement. Eleven students mentioned
increasing time with faculty members. Three students suggested making peer presentations
more interactive, either by having a faculty liaison or the presenter calling on other
students.
Discussion
A month-long remote course was successfully implemented during COVID-19 to promote
learning in ophthalmology. There was a significant increase in self-reported knowledge
for MS3s and MS4s. In addition, self-reported interest in ophthalmology increased
significantly for MS3s. The class performed significantly higher in ophthalmology
questions than UWorld Step 2 CK question bank users. Our study suggests remote electives
may have great potential to enhance student education, knowledge, and interest in
ophthalmology.
Overall, the course was well received. Open-response survey questions demonstrated
high student satisfaction with course organization and quality of teaching materials.
Students praised the utility of peer presentations and faculty discussions. An area
of improvement was standardizing faculty discussions. Though faculties were provided
with cases, they were encouraged to utilize their own materials to permit innovative
teaching. The feedback suggests a structured approach to learning is favored. It also
advocates for a centralized curriculum in the field. An open-source Web site with
prerecorded lectures, standardized slides, and questions for discussion can augment
our elective, as well as virtual learning across institutions. Additional course improvements
could be made via faculty feedback on student engagement.
Over half of the class participated in a televisit. Many students noted the number
of telehealth experiences should be increased to once or twice per week. This is not
surprising, as these observations were only offered in the last week due to technical
difficulties. Thus, each student was only able to observe a single visit. Pairing
students with clinicians at least a week prior to the visit can also enhance the experience.
Seven students chose not to observe a visit. This cannot be attributed to low self-reported
interest alone, as a stratified analysis showed no significant difference between
televisit participants and nonparticipants. This result may be due to scheduling conflicts
outside of the elective, as students were asked to sign up toward the end of the third
week. It is also possible that students were not quarantined in a space to conduct
a professional virtual visit, preferring self-study activities.
It is important to acknowledge concerns regarding trainee involvement in the virtual
ophthalmic examination. Teleophthalmology poses many challenges to ophthalmologists.
Lack of access to diagnostic equipment restricts patient management strategies. Most
providers do not have previous telehealth training, so some may not precept a learner
until they are comfortable with using the virtual platform. Due to similar reasons,
some patients may not wish to have a trainee in the virtual room. As the comfort levels
of patients and providers improve, so will the opportunities for trainees.
The lessons learned in this course are not only applicable to crises that require
transition to remote learning but also for contemporary medical education. The utility
of in-person lectures compared with online lectures has been debated.[12]
[13]
[14] Prior to COVID-19, many medical schools had witnessed a decline in student attendance,
likely in part due to increased emphasis on licensing examinations by residency programs.[15]
[16]
[17] In the ever-changing landscape of medical education, remote courses may become permanent
or combined with clinical experiences.[18]
[19]
[20] At our institution, for example, we created an in-person/telehealth hybrid model
for the MS3 ophthalmology curriculum. Though we have not offered the elective to a
second cohort, our course design offers an alternative to the traditional “away” rotation.
As the entire curriculum can be conducted remotely, it can aid students without ophthalmology
programs at their home institution. It can also provide mentorship and career development
opportunities to faculty, who wish to train the next generation of ophthalmologists.
Our study's strengths include its large sample size and current relevance of remote
medical education. To our knowledge, this is the first distance-learning 4-week course
in ophthalmology implemented during COVID-19. With a sample size of 18 students between
two academic years, we were able to collect substantial quantitative and qualitative
information on course effectiveness and areas of improvement.
The largest limitation of our study is the lack of a precourse survey. Our results
were acquired from post hoc reflection, limiting internal validity. However, comparison
of posttest results with UWorld users provides an objective assessment of student
performance and increases external validity. The test bank users may also be limited
as a control group. Most UWorld users test their ophthalmology knowledge with randomly
generated questions across multiple topics, compared with our students who took a
short ophthalmology-specific quiz. Additionally, our study may be subject to selection
bias. Given these students chose our particular elective, they were likely more interested
in ophthalmology compared with the medical student population. This is evident from
the moderate MS3 and high MS4 self-reported baseline interest. In spite of limitations,
the improvement in self-reported knowledge and interest underscores the success of
our curriculum.
Conclusion
In summary, we present a remote ophthalmology curriculum for medical students shown
to be successful during COVID-19. This curriculum may be helpful to educators worldwide,
who wish to introduce the field to students or integrate an online curriculum with
a clinical ophthalmology course. Studies of telehealth experience and objective assessment
are needed to further explore the potential of remote learning in ophthalmology, as
COVID-19 continues to transform medical education.
