Keywords
optics - flipped classroom - problem-based learning - ophthalmology - residency - medical education
Ophthalmologists must demonstrate competence in clinical optics to attain professional certification by the American Board of Ophthalmology. Unfortunately, the traditional lecture-based method of instructing ophthalmology residents in optics has anecdotally in our experience been perceived by learners and instructors to be both challenging and of low educational yield. Alternative active learning models such as the flipped classroom approach may help to address these concerns.[1]
The flipped classroom method involves learners performing independent study outside of the classroom before attending in-class sessions. During in-class sessions, learners participate actively by teaching the material covered during independent study to their peers.[1] Key elements of the flipped classroom approach were first introduced as a “peer instruction” method in 1991 by Eric Mazur at Harvard University.[2] Over the next few decades, variations in this curricular approach were popularized and applied to medical education as the “flipped classroom” method, although there did not exist clear evidence that this pedagogical approach would improve the competency of healthcare providers. More recently, there has been building evidence that flipped classroom curricula can be more effective than lecture format in some medical education settings.[1]
[3] However, meta-analyses thus far comparing the efficacy of flipped classroom and lecture-based medical curricula have not conclusively weighed in favor of the flipped classroom methodology.[1]
[3]
[4]
[5] It has been difficult to ascertain the impact of flipped classroom curricula on learners in part due to variability in both flipped classroom and research methodology among past studies.[4]
With regard to clinical optics, there have been advocates for an active learning, problem-focused approach to this subject matter as early as 1913.[6] However, the efficacy of an active educational format applied to clinical optics has not been directly examined. We aimed to test whether a flipped classroom approach to clinical optics education can be more effective than traditional lecture-based didactics in facilitating learner acquisition of clinical optics knowledge and improving their ability to apply this information. To our knowledge, this is the first investigation into the implementation of a flipped classroom format to clinical optics in a graduate medical education setting.
Methods
Ophthalmology resident clinical optics instruction at Oregon Health & Science University was delivered via traditional lecture-based didactics until a flipped classroom curriculum was implemented during the 2012 to 2013 academic year and continued thereafter. [Table 1] summarizes the major elements of the two curricula.
Table 1
Comparison of educational curricula
|
Curriculum type
|
Lecture-based curriculum
|
Flipped classroom curriculum
|
|
Focus
|
Instructor-centered
|
Learner-centered
|
|
Learner activities prior to in-class sessions
|
Learners provided with reference books and recommended reading
|
Learners provided with reference books, recommended reading, and online video lecture series. Learners complete assigned problem sets independently
|
|
Learner and instructor activities during in-class sessions
|
Lectures delivered by faculty instructors via projected slide presentations. Learners listen to lectures and are allowed to ask questions
|
Learners take turns teaching each other in small groups. Faculty provide oversight and guidance as needed
|
|
Faculty instructor time commitment and preparation
|
Instructors prepare lecture slide presentations
|
Instructors review predefined problem set and answer key
|
|
Academic years curriculum applied
|
AY 2008–09 to AY 2011–12
|
AY 2012–13 to AY 2015–16
|
|
Timing of in-class sessions
|
Fall (September–October)
|
Winter (January–February)
|
|
Mean number of in-class sessions per year
|
4.5
|
4.25
|
|
Mean number hours dedicated to in-class sessions per year
|
18.0
|
11.75
|
Abbreviation: AY, academic year.
Our lecture-based curriculum consisted of projected slide presentations developed and delivered by faculty. Residents were expected but not required to prepare for lecture by reading the appropriate sections from the Basic and Clinical Science Course (BCSC) reference books. Didactics were delivered in the fall from September to October.
Our flipped classroom approach to optics was structured as follows: 4 sequential problem sets were constructed, in total consisting of 100 optics questions, 42 of which are from ophthalmic optics and clinical refraction by Guyton et al.[7] Learners were asked to independently complete the appropriate set of optics questions prior to each of four mandatory in-class group sessions. Learners were provided with the following reference materials to utilize during their initial independent study period—BCSC Section 3: Clinical Optics, Ophthalmic Optics and Clinical Refraction by David Guyton, Review questions in ophthalmology, and the prerecorded optics video lecture series by David Hunter.[7]
[8]
[9]
[10] We instructed residents to prepare to teach the material covered in the problem sets to their peers. At each in-class session, residents took turns working through the assigned problems and teaching their peers within small groups of four to five learners organized by year of training. Faculty instructors supervised the discussions, answered questions, and corrected mistakes, but did not lead the teaching sessions. In class, flipped classroom sessions were delivered in late winter, from January to February.
We compared resident performance on the Ophthalmic Knowledge Assessment Program (OKAP) annual national in-service examination for the 4 years before and after the optics curriculum change. No other major curricular changes were implemented during this period of time. All OKAP examination scores from 2009 to 2016 were included with no exclusions. Data from each year included scores from 15 residents except 2009 (12 residents) and 2014 (14 residents). The scaled scores from the Optics, Refraction, and Contact Lens subsection were examined with scores from the 10 nonoptics subsections serving as controls. Descriptive statistical analysis was performed using Microsoft Excel (Redmond, WA). Unpaired, two-tail t-tests were performed on the scores from the 11 subsections of the OKAP examination, comparing scores before (2009–2012) and after (2013–2016) the optics curriculum change. Bonferroni correction was applied to adjust the p-value for statistical significance given the 11 subsection comparisons. This analysis was also repeated for each postgraduate year (PGY) class of residents.
Results
The Optics, Refraction, and Contact Lens mean subsection scores were found to be 50.4 ± 2.3 and 57.3 ± 2.5 (mean ± 95% confidence interval [CI]) before and after the optics curriculum change, respectively ([Fig. 1]). Optics, Refraction, and Contact Lens was the only subsection to show a statistically significant difference when comparing scores before and after the optics curriculum change (p = 0.00008, [Table 2]). Descriptive statistical analysis of all 11 OKAP examination subsection scores before and after the optics curriculum change is shown in [Table 2]. Bonferroni correction of the α to account for the multiple comparisons yielded a p value of 0.0046 necessary for statistical significance. Subgroup analysis of each PGY of training showed that PGY-2 and PGY-3 optics scores were higher with flipped classroom (55.9 ± 4.4 and 59.1 ± 4.1, respectively, mean ± 95% CI) compared with lecture-based curricula (46.7 ± 4.4 and 51.1 ± 3.7, respectively, mean ± 95% CI). This difference was statistically significant with p values of 0.0037 for PGY-2 and 0.0043 for PGY-3 residents. There was no statistically significant difference in optics subsection scores for PGY-4 residents when comparing flipped classroom and lecture-based curricula (53.3 ± 5.1 and 56.8 ± 4.1, respectively, mean ± 95% CI; p = 0.25).
Table 2
OKAP subsection performance
|
OKAP subsection
|
Before or after optics curriculum change
|
Mean score
|
Median score
|
SD
|
95% CI
|
Max
|
Min
|
Range
|
n
|
p-Value
|
|
1. General medicine
|
Before
|
49.51
|
49
|
9.1
|
2.4
|
67
|
29
|
38
|
57
|
0.0052
|
|
After
|
54.41
|
57
|
9.4
|
2.5
|
66
|
16
|
50
|
59
|
|
2. Fundamentals and principles of ophthalmology
|
Before
|
52.42
|
53
|
9.1
|
2.4
|
69
|
32
|
37
|
57
|
0.59
|
|
After
|
53.29
|
53
|
8.3
|
2.2
|
68
|
30
|
38
|
59
|
|
3. Optics, refraction, and contact lens
|
Before
|
50.37
|
51
|
8.7
|
2.3
|
67
|
32
|
35
|
57
|
0.000083
|
|
After
|
57.27
|
60
|
9.5
|
2.5
|
74
|
34
|
40
|
59
|
|
4. Ophthalmic pathology and intraocular tumors
|
Before
|
53.72
|
54
|
7.0
|
1.8
|
68
|
39
|
29
|
57
|
0.10
|
|
After
|
55.92
|
57
|
7.4
|
1.9
|
67
|
23
|
44
|
59
|
|
5. Neuro-ophthalmology
|
Before
|
53.30
|
53
|
8.5
|
2.2
|
72
|
28
|
44
|
57
|
0.074
|
|
After
|
56.19
|
57
|
8.8
|
2.3
|
71
|
27
|
44
|
59
|
|
6. Pediatric ophthalmology and strabismus
|
Before
|
52.33
|
54
|
8.5
|
2.3
|
68
|
28
|
40
|
57
|
0.18
|
|
After
|
54.51
|
56
|
8.8
|
2.3
|
67
|
24
|
43
|
59
|
|
7. Orbit, eyelids, and lacrimal system
|
Before
|
54.16
|
54
|
6.6
|
1.8
|
67
|
35
|
32
|
57
|
0.88
|
|
After
|
53.93
|
56
|
9.5
|
2.5
|
67
|
7
|
60
|
59
|
|
8. Cornea, lens, and external disease
|
Before
|
51.74
|
51
|
8.3
|
2.2
|
67
|
25
|
42
|
57
|
0.25
|
|
After
|
53.47
|
54
|
8.0
|
2.1
|
70
|
30
|
40
|
59
|
|
9. Intraocular inflammation and uveitis
|
Before
|
52.02
|
52
|
7.0
|
1.9
|
64
|
23
|
41
|
57
|
0.034
|
|
After
|
55.22
|
56
|
8.9
|
2.3
|
70
|
25
|
45
|
59
|
|
10. Glaucoma
|
Before
|
52.35
|
52
|
9.2
|
2.4
|
68
|
31
|
37
|
57
|
0.040
|
|
After
|
55.97
|
59
|
9.6
|
2.5
|
70
|
20
|
50
|
59
|
|
11. Retina and vitreous
|
Before
|
53.93
|
56
|
7.7
|
2.0
|
68
|
33
|
35
|
57
|
0.71
|
|
After
|
53.34
|
55
|
9.3
|
2.4
|
67
|
18
|
49
|
59
|
Abbreviation: CI, confidence interval; OKAP, Ophthalmic Knowledge Assessment Program; SD, standard deviation.
Fig. 1 Ophthalmic Knowledge Assessment Program (OKAP) examination scores before and after implementation of a flipped classroom optics curriculum. Resident OKAP examination performance for the 4 years before and after changing the optics curriculum from a lecture-based to flipped classroom curriculum (mean ± 95% CI). See [Table 2] for full-length OKAP subsection titles. Asterisk symbol denotes statistically significant difference at p = 0.00008. CI, confidence interval.
Discussion
To date, there has been minimal research into the application of alternative educational methodologies such as the flipped classroom format to ophthalmology education.[11]
[12]
[13] Our data show that transitioning from traditional lectures to a flipped classroom curriculum for clinical optics was associated with a statistically significant increase in mean OKAP examination score for the Optics, Refraction, and Contact Lens subsection. This suggests that the flipped classroom format may be more effective than lectures for facilitating ophthalmology resident clinical optics knowledge and ability to apply this knowledge in a standardized test setting. Standardized test scores are relevant, since a written qualifying examination (a standardized test similar in nature to the OKAP examination) is required for American Board of Ophthalmology certification, and the board pass rate for graduates is in turn a key metric monitored by the Accreditation Council for Graduate Medical Education (ACGME) ophthalmology residency review committee for the purpose of residency program accreditation.[14]
[15]
The flipped classroom format as a form of active learning has been shown to positively influence learner engagement, motivation, and satisfaction.[4]
[16]
[17]
[18]
[19] Our flipped classroom optics curriculum places learners in a more active role, requiring them to both complete problem sets independently and teach their colleagues. Of note, our learners were also given a choice of reference materials to utilize during the pre-class self-study interval. Trainee learning preferences may differ, with some favoring prerecorded video lectures and others preferring reference books. While the average in class time dedicated to our flipped classroom curriculum was less than that of our lecture-based curriculum, flipped classroom learners were required to spend additional time outside the classroom to complete problem sets. We did not quantify the number of hours spent by residents outside of the classroom, but based on our own experience completing the problem sets, we estimate the total time spent by learners in and out of the classroom to be roughly similar between the two curricular approaches. Feedback from learners suggests that the average time spent completing the self-study portion of the flipped classroom curriculum decreases with each year of experience.
Interestingly, PGY-2 and PGY-3 but not PGY-4 residents exposed to our flipped classroom curriculum performed significantly better on the OKAP examination than respective PGY counterparts exposed to a lecture-based optics curriculum. This data suggests that the flipped classroom format benefited inexperienced optics learners more so than those with 2 years of pre-existing optics examination experience. Of course, subgroup analysis is more prone to type 1 statistical error given the smaller sample size.
Anecdotally, we found our problem-based optics curriculum to be less burdensome and more enjoyable for instructors. Faculty were no longer required to spend time generating slide presentations for the optics lecture series. Providing faculty with problem set answer keys allowed them to efficiently prepare for in-class sessions. Additionally, faculty time commitment required for in-class educational sessions was less with the flipped classroom curriculum than with traditional lectures. While a similar group of optometry and ophthalmology teaching faculty was utilized in this study both before and after the curriculum change, we did find that the flipped classroom format with pre-constructed questions and answer key was more conducive to the recruitment of faculty instructors. The flipped classroom format likely lowers the barrier to participation, especially for faculty without a clinical practice that focuses on optics. This may be helpful for residency training programs that struggle to recruit faculty to teach optics.
This study is subject to several limitations including those associated with a retrospective, nonrandomized study design based on data from a single institution. A limitation of our study involves the sampling error that can occur due to the relatively few questions on any one OKAP examination subsection. As noted in the OKAP user guide: “because each yearly exam is comprised of a different set of test items, not every clinical point can be covered in every year. […] Because subtests necessarily must contain fewer items than the test as a whole, subtest scores are somewhat less reliable and must be interpreted with more caution.”[20]
Another limitation of our study is that we are unable to separate the contributions of the different components of our flipped classroom curriculum. The required self-study problem set, learners as teachers small group discussion format, and the timing of curriculum prior to testing were all factors that may have contributed to the performance improvement observed.
While we do not have data to directly test whether the required problem set alone would be sufficient to improve optics scores, there is extensive literature suggesting that collaborative learning yields superior educational outcomes relative to individualistic learning.[19] Additionally, our data suggests that simply providing or improving access to problem sets is not sufficient to optimize examination performance. OKAP-style practice questions were made available to all residents throughout the study period, including those found in review questions in ophthalmology and multiple editions of the BCSC series.[9]
[10] Also of note, ophthoquestions.com, an online resource that could potentially increase learner access to OKAP-style practice questions, was introduced in September 2011. Our program has never paid for this online question bank for our trainees and we do not track those that utilize it; however, ophthoquestions.com company representatives report that 95% of ophthalmology trainees utilize their services for OKAP preparation.[21] No statistically significant difference was identified when comparing optics subsection scores before and after the introduction of ophthoquestions.com (and before the introduction of the flipped classroom curriculum). Similarly, no statistically significant difference in nonoptics subsection test scores was identified when comparing scores prior to and after the introduction of ophthoquestions.com. This suggests that improved online access to optics questions as well as nonoptics questions does not significantly improve OKAP scores.
Our dataset is not sufficient to fully evaluate whether the temporal shift of optics curriculum to more closely precede the OKAP test (delivered in March) was influential, but suggests that the effect, if present, was small. There was no statistically significant difference between mean subsection exam scores for curricula presented in the fall versus late winter (51.7 ± 2.0 and 52.8 ± 2.3, respectively, mean ± 95% CI; p = 0.23).
While our research shows that the flipped classroom format can be applied in an advantageous manner to improve ophthalmology resident optics standardized test scores, further study will be needed to determine whether this curricular shift translates to an improved ability to apply optics knowledge to real-world clinical practice. And finally, while these results may be applicable to the optics portion of the ophthalmology residency curriculum, it is unclear whether flipped classroom would also be more effective than traditional lectures for other ophthalmologic areas of study.
Despite these limitations, given the relative advantages of flipped classroom optics learning over traditional lecture-based curricula, a flipped classroom approach is a reasonable option for ophthalmology training programs seeking to improve resident performance in clinical optics.
