Medical Student Ophthalmic Knowledge Proficiency after Completing a Clinical Elective or an Online Course

Keywords

online course - clinical elective - clinical rotation - ophthalmology medical education

Introduction

Medical school curricula are challenged with transmitting an ever-increasing amount of information to students.[1] As a result, emphasis has necessarily shifted toward mainstream topics, such as internal medicine and general surgery, and away from the more nonmainstream, specialized disciplines. Yet, the health care system designates primary care providers (PCPs) as gatekeepers of nonmainstream care, so they must acquire enough knowledge of medical specialties to be able to treat and triage appropriately.[2]

Among the nonmainstream medical specialties is ophthalmology. Equipping medical students with an adequate knowledge of ophthalmology is important because ophthalmic symptoms are common among patients consulting PCPs.[3] [4] Even so, a survey of program directors in family, internal, and emergency medicine found that medical students and physicians are inadequately trained to deal with the initial management and appropriate referral of the most basic ophthalmic problems.[2] Meanwhile, ophthalmology education in medical school is being marginalized as a consequence of reduced access to ophthalmic clinical skills training in medical curricula, a trend that is increasing because of constraints in time, funding, teachers, and resources. Liesegang et al[5] listed the following three reasons for this marginalization: competition among medical specialties desiring time to teach, lack of commitment of ophthalmology teachers in medical schools, and greater availability of community ophthalmologists and optometrists, so that PCPs view substantial knowledge of the eye and its diseases as unnecessary.

To combat this problem, online teaching and “flipped classrooms” teaching have emerged as alternatives. These approaches have become even more relevant to undergraduate medical education in the novel coronavirus disease 2019 (COVID-19) pandemic period, during which students have had to forgo in-person clinical electives in search of other means of learning. In the pre–COVID-19 era, Petrarca et al[6] found that among 245 British medical students who were taught ophthalmology concepts via online modules or in-person lectures, those who had completed the online modules performed better on a 100-question written examination than those who had attended traditional lectures. Students who took the online modules were also more satisfied with the course.[6] Tang et al,[7] Lin et al,[8] and Diel et al[9] also found that, compared with a traditional curriculum of didactic lectures, flipped classrooms in ophthalmology do not decrease test scores but do increase student satisfaction.

Although many studies have shown the benefit of using online and flipped ophthalmology courses in place of curricula with didactic lectures, no study has compared an in-person clinical experience to the use of these approaches. Accordingly, we designed a study to assess the performance on a multiple-choice ophthalmic knowledge proficiency examination of medical students at the University of Michigan Medical School who had completed a 2-week in-person elective in the ophthalmology clinics and operating rooms, together with an online course, as compared with the performance of medical students who had completed an online course alone. We also surveyed both groups of students before and after the course to assess pertinent group characteristics and perceptions.

Methods

Subject Selection

All subjects were third or fourth year medical students in the 2019 to 2020 academic year matriculating at the University of Michigan. They had already completed the seven core clinical electives (internal medicine, family medicine, psychiatry, neurology, surgery, obstetrics and gynecology, and pediatrics) and the U.S. Medical Licensing Examination (USMLE) Step 1 examination. We did not know whether the students had already completed the USMLE Step 2 Clinical Knowledge or Clinical Skills examinations.

Courses

Subjects were voluntarily enrolled through the online medical student course catalog for one of two elective courses. The choices were an online course alone (Online Only) or a combined 2-week clinical elective and online course (Clinical + Online). The Online Only group was targeted at medical students who were contemplating a career in fields other than ophthalmology. They were required to obtain satisfactory scores on the quiz portion of an online course entitled “The Eyes Have It” (TEHI), consisting of nine didactic modules and quizzes. There were no lectures or clinical duties. The combined clinical and online elective (Clinical + Online) group was designed for students considering ophthalmology as a career. They enrolled in the standard 2-week clinical elective offered by the University of Michigan, Department of Ophthalmology and Visual Sciences, which included attending 3 hours of in-person lecture and spending 80 hours in the clinics or operating rooms, as well as obtaining satisfactory scores on the quiz portion of TEHI. Both groups were permitted to use outside teaching materials and were required to complete an ophthalmic knowledge proficiency written examination.

The online course, TEHI,[10] is a free web site in the public domain coauthored by two of the investigators (J.D.T. and J.J.A.H.). It consists of nine didactic modules covering eye symptoms and signs, red eye, eye trauma, ophthalmic and systemic conditions, side effects of medications, the ophthalmic screening examination, ophthalmoscopic abnormalities, and anatomic pathways of the eye and visual system. All enrollees were required to complete the nine multiple-choice quizzes, one for each of the nine TEHI modules, with a total of 140 questions. Each question included a written explanation for the correct and incorrect answer choices, a format similar to that found in commercial question banks that medical students use to prepare for the USMLE. Students in the Clinical + Online and the Online Only groups were required to score at least 80% on each quiz to receive credit for the course. They were permitted to take each quiz as many times as needed to obtain the 80% passing score. The TEHI authors had no input into the ophthalmic knowledge proficiency written examination.

Ophthalmic Knowledge Proficiency Written Examination

Serving as the criterion of ophthalmic knowledge proficiency, this 50-question multiple-choice examination was compiled from ophthalmology teaching textbooks and written examinations used at other academic institutions (Appendix). It was assembled by one of the investigators (A.D.K.), who had no input into the TEHI modules or quizzes and had not studied their content. The examination was reviewed for its pertinence to nonophthalmic medical practice by an emergency medicine resident physician, two pediatric resident physicians, and an internal medicine resident physician. Clinical + Online and Online Only students completed the examination on the Canvas teaching online platform (Instructure, Salt Lake City, UT). All students were required to attest to the University of Michigan honor code prior to taking the examination.

Precourse and Postcourse Surveys

Students in both groups completed two surveys. The precourse survey was designed to provide information about whether students were in the career exploration phase of electives during the time they participated in the courses, their reasons for enrolling in the courses, and whether they had had any previous experience in shadowing an ophthalmologist or doing ophthalmic research. It also collected baseline levels of their intentions to pursue a career in ophthalmology and their confidence in managing ophthalmic problems. The postcourse survey was designed to elicit the amount of time the students spent on the course, outside resources they used during the course, a change in their intention to pursue a career in ophthalmology, and a change in their confidence in managing ophthalmic problems. Survey responses assessing baseline and change in intention to pursue a career in ophthalmology and confidence in managing ophthalmic problems were graded on a 5-point Likert's scale with 1 = definitely not/strongly disagree and 5 = definitely/strongly agree ([Tables 1], [2]).

Table 1 Survey eliciting pertinent characteristics of the two student groups

		Clinical + Online n (%)	Online Only n (%)	p-Value
		n = 20	n = 59
	Mean age (range)	26.0 (23–30)	26.9 (24–27)	0.10
	Gender (% women)	8 (40)	33 (56)	0.33
Precourse	1. Career exploration phase of medical school	16 (80)	10 (17)	<10−4
	2. Why are you taking this course?
	Explore a career in ophthalmology	15 (75)	0 (0)	<10−4
	Strengthen clinical understanding of ophthalmology	18 (90)	55 (93)	0.64
	Have exposure to ophthalmology faculty	11 (55)	0 (0)	<10−4
	Flexibility for travel or studying for USMLE step 2	2 (10)	31 (53)	<10−4
	3. Previous exposure to ophthalmology	11 (55)	7 (12)	<10−4
Postcourse	4. Use of outside teaching resources	10 (50)[a]	5 (10)[b]	0.002
	5. Time devoted to completing TEHI modules and quizzes (h)
	Total	20.5	19.5	0.59
	Time devoted to completing TEHI modules	12.5	12.5	0.99
	Time to complete TEHI quizzes	8.8	7.0	0.33

Abbreviations: TEHI, the eyes have it; USMLE, U.S. Medical Licensing Examination.

^a Clinical + Online students used the American Academy of Ophthalmology (AAO) Web site,[12] Basic Ophthalmology: Essentials for Medical Students,[13] and OphthoBook.[14]

^b Online Only students used notes from the pre-clinical ophthalmology block, American Academy of Family Physician (AAFP) guidelines,[15] and UpToDate.[16]

Table 2 Survey eliciting changes in pertinent student perceptions after completing the course

		Clinical + Online n = 20	Online Only n = 59	Difference	p-Value between groups
1. I will be pursuing a career in ophthalmology	Precourse	3.00	1.12	1.88	<10−4
	Postcourse	3.50	1.14	2.36	<10−4
	Change	0.50	0.02	0.48	<10−4
2. I have an appropriate understanding of ophthalmic diseases expected of non-ophthalmologists	Precourse	2.55	2.14	0.41	0.064
	Postcourse	4.15	3.85	0.30	0.15
	Change	1.60	1.71	−0.11	0.67
3. I know which ophthalmic problems can be handled by a general practitioner	Precourse	2.35	2.34	0.01	0.97
	Postcourse	4.10	3.97	0.13	0.45
	Change	1.75	1.63	0.12	0.65
4. I know when an ophthalmology consult is appropriate	Precourse	2.50	2.63	−0.13	0.59
	Postcourse	4.20	4.20	0.00	0.98
	Change	1.70	1.57	0.13	0.62
5. I know how urgently each ophthalmic presentation needs to be referred to an ophthalmologist, if at all	Precourse	2.30	2.24	0.06	0.81
	Postcourse	4.10	4.15	−0.05	0.76
	Change	1.80	1.91	−0.11	0.68
6. I know how an ophthalmologist can contribute to the overall care of patients with nonophthalmic diseases (diabetes mellitus, rheumatoid arthritis)	Precourse	3.50	3.27	0.23	0.41
	Postcourse	4.85	4.47	0.38	0.02
	Change	1.35	1.20	0.15	0.61

Note: Results are averages of responses scored on a Likert scale (1 = definitely not/strongly disagree, 5 = definitely/strongly agree). The vertical column titled “Difference” represents Clinical + Online minus Online Only survey responses for each question. The “Change” rows represent postcourse minus precourse survey responses.

Four students, three from the Online Only group and one from the Clinical + Online group, did not answer the survey questions and were excluded from the study.

Statistical Analysis

We performed bivariate analyses using IBM SPSS Version 26 (Armonk, NY). Chi-square tests were used to compare categorical survey responses between the two groups. Fisher's exact tests were used when there were small sample sizes among comparison groups. Unpaired, two-sample t-tests were performed using the DataAnalysis ToolPak by Microsoft Excel (Seattle, WA) for comparing continuous outcomes from the survey results and examination scores between the two groups. The Holm–Bonferroni method[11] was used to adjust for multiple comparisons when comparing performance between groups on single ophthalmic knowledge proficiency examination items. Lastly, correlation coefficients were computed using the DataAnalysis ToolPak by Microsoft Excel (Seattle, WA).

Ethics

The University of Michigan IRB reviewed the study and determined it to be exempt (HUM00161915). Data were collected anonymously and students had the option to not complete the surveys.

Results

During the study period, 20 students completed the Clinical + Online course and 59 students completed the Online Only course.

Proficiency

On the 50-question ophthalmic knowledge proficiency written examination, the Clinical + Online group performed slightly better than the Online Only group (mean [standard deviation (SD)] score, 86.3 (5.6) vs. 83.0% (3.7), p = 0.004), approximately equivalent to a 2-question difference in performance between the two groups. The following two questions stood out as those that most separated the performance of the Clinical + Online and Online Only students:

1. A 45-year old man sustains an alkali injury to his left eye while working. He rushes to the emergency room and you proceed to irrigate his left eye until? Correct answer: the pH neutralizes (Clinical + Online with 95% students answering correctly, Online Only with 59%, p = 0.003).

2. A 40-year old man reports that he has intermittent blurry vision that improves with blinking. On further questioning, he says that it only occurs at the end of a work day after using the computer for many hours. The most likely diagnosis is? Correct answer: dry eye syndrome (Clinical + Online with 100% answering correctly, Online Only with 76%, p = 0.016).

The answers to these two questions were not addressed in TEHI. If the average test scores between the two groups are compared after excluding these two questions, there is no difference in the examination scores of the two groups (85.8% for Clinical + Online vs. 85.4% for Online Only, p = 0.069).

Surveys

There were differences in the characteristics of the Clinical + Online and Online Only groups ([Tables 1], [2]). First, Clinical + Online students were much more likely to be in the career exploration phase (80%) of their medical education than were the Online Only students (17%). A nearly equal percentage of the Clinical + Online students (90%) and Online Only students (93%) took the course to strengthen their understanding of ophthalmology. Fifteen (75%) Clinical + Online students took the course to explore a career in ophthalmology, whereas no Online Only students did so for that purpose. Eleven (55%) Clinical + Online students took the in-person course to gain exposure to ophthalmology faculty, whereas no student in the Online Only group sought that exposure. Thirty-one (53%) Online Only students took the course for flexibility to travel or study for the USMLE step-2 as compared with 10% of students in the Clinical + Online group. A higher proportion of Clinical + Online than Online Only students had had previous clinical or research exposure to ophthalmology.

Both groups spent an average of approximately 20 hours completing TEHI modules (20.5 for the Clinical + Online group and 19.5 for the Online Only group). However, the Clinical + Online group had an additional 83 hours of requirements, including 48 hours in clinic, 32 hours in the operating room, and 3 hours in lecture. A larger proportion of students in the Clinical + Online group (50%) used additional resources than in the Online Only group (10%, p = 0.002). Clinical + Online students used more traditional teaching resources than did the Online Only students ([Table 1]).

In the answer to the postcourse survey question designed to elicit whether students had shifted in their intention to pursue a career in ophthalmology, there was significant increase in the Clinical + Online cohort and little change in the Online Only cohort. In the answers to questions aimed at student-perceived preparedness in dealing with ophthalmic problems, there were no significant differences in the changes from precourse to postcourse responses between the two groups. ([Table 2]).

When all 79 students were analyzed as a single group, a linear regression showed that the self-reported number of hours devoted to the course was weakly correlated to the final examination score (r = 0.28, p = 0.013). Similarly, interest in pursuing a career in ophthalmology was weakly correlated to final examination score (r = 0.22, p = 0.046).

Discussion

Students in the Clinical + Online group performed slightly better on the written ophthalmic knowledge proficiency written examination than did the students in the Online Only group, approximately equivalent to 2 out of 50 questions. This difference may have been influenced by the content of these two questions, which were not covered adequately, if at all, in the TEHI modules. Once these questions were removed from the analysis, the difference in average examination scores between the two groups was not significant, suggesting noninferiority in the performance of the Online relative to the Clinical + Online group. The ability of the Clinical + Online students to answer those two questions correctly may be attributable to their being more interested in pursuing a career in ophthalmology and in reading more outside resources.

This small difference in knowledge proficiency must be balanced against the substantial increase in time devoted to the clinical rotation. It must also be balanced against the fact that student responses showed no difference between the two groups in regard to their self-perceived comfort in managing common ophthalmic problems. The two groups expressed an equal baseline comfort level and reported an equal increase in comfort managing common ophthalmic problems after the course.

A critical problem in the training of medical students on ophthalmology clinical electives is the lack of exposure to a broad range of ophthalmic conditions. Students must settle for encountering only those conditions that appear during their rotation time. Even when those conditions are encountered, students are unlikely to view physical examination manifestations in detail, as they are accessible only through biomicroscopy or indirect ophthalmoscopy, examination techniques that students cannot master in a short time. In this respect, there is a clear advantage to online teaching tools, which can cover a wider gamut of the important ophthalmic problems and display them to full advantage. Students can absorb the information at a personalized pace that is optimal for their learning and receive continuous feedback via short quizzes. They can complement their time during the course with other endeavors, which is helpful considering the increasing volume of information that medical students are required to learn. The exigencies of the COVID-19 pandemic brought these advantages to the fore. During that period, 25 additional students signed up for the Online Only option.

We acknowledge that an online course cannot adequately assess a student's ability to execute commonly used ophthalmic clinical examination techniques, including the testing of visual acuity, confrontation visual fields, pupil reactions, ocular motility and ocular alignment, eversion of the eyelid, direct ophthalmoscopy, and the slit lamp examination. These specialized techniques demand considerable monitored practice. To that point, we have observed that medical students participating in clinical electives often do not have much chance to practice these techniques and are rarely evaluated for their proficiency. This issue, which is particularly problematic in instrument-dominated fields such as ophthalmology and otolaryngology, is difficult to rectify, especially as students who are increasingly shuffled between busy clinics.

Limitations

A limitation of this study is the small sample size, which precluded more in-depth statistical analysis and comparison between the two groups. The Online Only group was much larger than the Clinical + Online group. In addition, the validity of the results hinges on whether the ophthalmic knowledge proficiency examination is a reasonable stand-in for the ability to handle ophthalmic problems. Even so, this study's outcomes support the proposition that an online course provides an acceptable means of learning as gauged by knowledge proficiency. The clinical elective might be more appropriate for the small proportion of students (typically 5–8% at this medical school) who have already decided on ophthalmology as a career, or are interested in exploring that possibility, as it allows them to form personal relationships with ophthalmology trainees and faculty ophthalmologists. For the remaining 92 to 95% of medical students who will not choose ophthalmology as a career, an online course should be more than adequate and conserve valuable time. The online-only substitute would also be applicable to other clinical subspecialties, such as otolaryngology and dermatology, which have similar learning impediments and are suited to a well-designed online course.

Conclusion

If the results of our study can be replicated in a larger cohort, they would suggest that an online course might be a fitting alternative to a clinical elective in the transmission of basic knowledge, especially in a subspecialty of medicine. The clinical exposure could then be devoted more to the transmission of examination skills, a goal that cannot be achieve as well in an online curriculum.

• References

• 1 Velthuis F, Varpio L, Helmich E, Dekker H, Jaarsma ADC. Navigating the complexities of undergraduate medical curriculum change: change leaders' perspectives. Acad Med 2018; 93 (10) 1503-1510
• 2 Stern GA. The Association of University Professors of Ophthalmology Education Committee. Teaching ophthalmology to primary care physicians. Arch Ophthalmol 1995; 113 (06) 722-724
• 3 Adar T, Levkovich I, Castel OC, Karkabi K. Patient's utilization of primary care: a profile of clinical and administrative reasons for visits in Israel. J Prim Care Community Health 2017; 8 (04) 221-227
• 4 Shah M, Knoch D, Waxman E. The state of ophthalmology medical student education in the United States and Canada, 2012 through 2013. Ophthalmology 2014; 121 (06) 1160-1163
• 5 Liesegang TJ, Hoskins Jr. HD, Albert DM. et al. Ophthalmic education: where have we come from, and where are we going?. Am J Ophthalmol 2003; 136 (01) 114-121
• 6 Petrarca CA, Warner J, Simpson A. et al. Evaluation of eLearning for the teaching of undergraduate ophthalmology at medical school: a randomised controlled crossover study. Eye (Lond) 2018; 32 (09) 1498-1503
• 7 Tang F, Chen C, Zhu Y. et al. Comparison between flipped classroom and lecture-based classroom in ophthalmology clerkship. Med Educ Online 2017; 22 (01) 1395679
• 8 Lin Y, Zhu Y, Chen C. et al. Facing the challenges in ophthalmology clerkship teaching: Is flipped classroom the answer?. PLoS One 2017; 12 (04) e0174829
• 9 Diel RJ, Avdic A, Kemp PS. Flipped ophthalmology classroom: a better way to teach medical students. J Acad Ophthalmol 2020; 12: e104-e109
• 10 Trobe JD. The Eyes Have It. Available at: http://kellogg.umich.edu/theeyeshaveit . Accessed September 28, 2020
• 11 Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. BMJ 1995; 310 (6973): 170
• 12 American Academy of Ophthalmology EyeWiki. Welcome to EyeWiki, the Eye Encyclopedia written by Eye Physicians & Surgeons. Available at: eyewiki.aao.org . Accessed September 28, 2020
• 13 Allen RC, Harper RA. Basic Ophthalmology: Essentials for Medical Students. 10th ed.. San Francisco, CA: American Academy of Ophthalmology; 2016
• 14 Root T. OphthoBook. 2nd ed.. Middletown, IN: CreateSpace Independent Publishing; 2018
• 15 American Association for Family Practitioners Practice Guidelines. Clinical Practice Guidelines. Available at: https://www.aafp.org/patient-care/browse/type.tag-clinical-practice-guidelines.html . Accessed September 28, 2020
• 16 UpToDate. Available at: https://www.uptodate.com/home . Accessed September 28, 2020