Evaluating the Efficacy of Microsurgical Training Methods in Ophthalmology Education: A Systematic Review and Meta-analysis

Abstract

Objective The objective of our paper is to review all of the relevant literature in ophthalmology microsurgical education and identify which teaching methodologies were most effective.

Methods A systematic review and meta-analysis of the literature was conducted. Electronic databases, including Ovid MEDLINE, Cochrane CENTRAL, and EMBASE, were searched with preset terms. The search was through December 11, 2019. Eligibility criteria included studies with sufficient data for analyzing associations between surgical teaching techniques and success rates in surgical skills and the organization of the intervention as a microsurgical skills course, curriculum, or program. The articles were independently reviewed by two authors. Each included study was evaluated for quality using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach and risk of bias using the Cochrane Collaboration's tool for assessing bias.

Data Extraction and Synthesis Data extraction was performed by two reviewers and disagreements were checked by a third reviewer. A random-effects analysis was used to pool the outcomes of studies.

Main Outcomes and Measures Outcomes included time for completion of surgical task, level of preparedness, competency score, and number of surgeries with complications.

Results A total of 439 studies were reviewed and 13 studies (n = 8,790 surgical cases; n = 115 trainees) were included in the meta-analysis. Excluded articles studied cataract simulation training as the primary intervention or were not related to ophthalmology. All pooled results demonstrated a positive association with surgical outcomes; however, video-based education (standardized mean difference [SMD] = 2.49 [95% confidence interval (CI): 0.36–4.63]; four effects [four studies]; n = 69; I ² = 90%) and stepwise teaching method (odds ratio [OR = 3.84 [95% CI: 2.66–5.55]; six effects [six studies]; n = 6,968; I ² = 39%) interventions were the most favorable.

Conclusion and Relevance The following five interventions evaluated in this paper were found to be effective methods of improving performance outcomes in ophthalmic microsurgery: (1) didactic lectures, (2) video-based education, (3) surgical wet-laboratory, (4) stepwise method, and (5) direct supervision and feedback. Our meta-analysis concludes that video-based education and stepwise teaching interventions are the most effective methods for a microsurgical ophthalmology training curriculum. Combining the strengths of the interventions analyzed in this study should be considered when implementing and adjusting ophthalmic surgical skills curriculums.

Keywords

microsurgery - surgical skills - ophthalmology training - residency education - surgical curriculum - training program - surgical education - surgical teaching

Surgical skills' curriculums are essential in training for surgically competent ophthalmologists, thereby also improving patient safety and outcomes in ophthalmic surgery. Ophthalmic surgical expertise requires trainees to master a diverse set of surgical skills including general surgical techniques and principles, microsurgical skills and instrumentation, and laser surgery expertise. With complex new surgical techniques consistently being introduced in this field, there is a need for efficient and flexible educational methods. The nature of ocular surgical procedures allows limited direct interactions between the supervising attending and resident, and surgical education at the level of the junior resident often results in the trainee learning through observation. There has been significant study into alternative surgical teaching techniques outside of the operating room to improve resident satisfaction and competence in surgical skills.

Wet-laboratory training is one of the most widely used teaching techniques in surgery. In the wet laboratory, residents are able to develop hand–eye coordination and learn the steps for surgical procedures in a systematic and low-stress setting. Residents have shown increased technical proficiency and enhanced confidence after completing a structured wet-laboratory curriculum.[1] Specifically for ophthalmology, animal and model eyes have been used to train residents.[2] Studies have illustrated a significant decrease in the frequency of intraoperative complications after the introduction of wet-laboratory training. Previous studies are predominantly retrospective studies, most of which do not control for innate skill levels of each resident, and many lack quantitative methods to measure residents' confidence levels.

The stepwise method is a teaching technique that involves learning a surgical procedure in defined steps.[3] In some studies, this method was paired with deliberate practice which aims to improve an individual's performance through strategic and highly focused sessions.[3] Although these studies showed that both methods paired together contributed to increasing residents' confidence, there were limited conclusions made regarding the success of either individually.

A variety of other surgical teaching methods have been explored which yielded promising results. Residents who were under supervision of an attending surgeon who directly observed their procedural skills and provided immediate feedback reported increased satisfaction and improvement of their skills.[4] Other literature has looked into utilizing more video-based surgical teaching modules and video review in place of traditional didactic lectures.[5] [6] Live teaching has the benefit of incorporating nonverbal two-way communication between the lecturer and trainee, and opportunity for real-time question and answer. Video teaching offers trainees the ability to rewind and replay. However, there is uncertainty about which of these teaching techniques is most effective. To our knowledge, no previous study has provided an overarching statistical analysis of all the evidence produced from these studies as they relate to ophthalmic surgical skills education.

Our meta-analysis aims to extract valid evidence to elucidate the relationship between specific educational interventions and surgical competency. We hope to fill in the gaps of knowledge within individual studies and draw connections between them.

Project Objectives

We conducted a systematic review and meta-analysis of surgical teaching to establish and evaluate program adjustments to the microsurgery training curriculum.

This study aimed to (1) extract valid evidence for effective surgical skills training techniques and assessments from ophthalmology surgical training research articles, and (2) identify relevant and feasible surgical skills modules which can be incorporated into an existing microsurgery training curriculum

Methods

Scope of Review and Institutional Review Board Approval

This review addressed five key questions, each exploring the effectiveness of one of the following educational interventions: (1) didactic lectures, (2) video-based education, (3) wet laboratory curriculum, (4) direct supervision and feedback from an attending surgeon, and (5) stepwise method. The Meta-Analysis of Observational Studies in Epidemiology (MOOSE) guidelines were followed to report the data. The protocol was approved by the University of Maryland Baltimore (UMB) Institutional Review Board (IRB) and determined to be exempt.

Literature Search

Electronic databases including Ovid MEDLINE, Cochrane CENTRAL, and EMBASE were used to identify cohort and case-control studies that investigated surgical teaching. The following search terms were used: (1) “Microsurgery,” (2) “Ophthalmolog*,” surgery (3) “Cataract surgery,” (4) “Cataract extraction,” (5) “Eye surgery,” (6) “Ocular surgery,” (7) “Refractive surgery,” (8) “Glaucoma surgery,” (9) “Ophthalmologic Surgical Procedures,” (10) “Surgical Training,” (11) “Surgical Curriculum,” (12) “Surgical Education,” (13) “Surgical Skills,” (14) “Residency Education.” The searches were conducted on December 11, 2019 (see [Supplemental Material S1], Complete Search Strategy [available in the online version]). The database searches were supplemented with manual searches in PubMed. The Covidence software (Veritas Health Innovation Ltd., Melbourne, Australia) was used for management of the screening process.

Eligibility Criteria

An initial screening of titles and abstracts was first performed to identify possible relevant studies. The full texts of identified studies were examined. Studies were considered eligible if they met the following criteria: (1) use of a cohort, case-control, or cross-sectional study design; (2) the study population were of ophthalmology residents, medical students, or other ophthalmology trainees; (3) provision of sufficient data for analyzing associations between surgical teaching techniques and resident satisfaction and success rates in surgical skills; (4) the study relates to ophthalmology and microsurgery surgical skills; and (5) the intervention in the study was organized as a microsurgical skills course, curriculum, or program. Studies in which surgical simulators, such as EYESI, acted as the primary intervention were excluded, as previous systematic reviews have extensively investigated this specific intervention.[7] [8] [9]

Study Selection

Articles were screened independently by two authors (G.N and J.P.). If necessary, a third author (J.A) resolved any disagreements. Duplicate articles were removed from the initial database search and the list of references were managed with Covidence. We did not require blinding of assessors.

Data Extraction

Covidence was used as an initial organizational tool to extract the interventions, outcomes, and population characteristics of each study. A Google Sheets spreadsheet (Google LLC, Mountain View, CA) was then used to categorize the articles by teaching methods used as follows: wet laboratory, didactic lectures, stepwise method, feedback/supervision, and video-based education. Outcomes were subcategorized as reduced complications, improved competency score, reduced time for completion, and increased level of preparedness. Data extraction was performed by two authors (G.N and J.P.) and reviewed by a third author (J.A).

Assessment of Quality and Risk of Bias

The quality of the selected studies was assessed using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach. This assessment system classifies the quality of the body of evidence in the study as one of four levels: high, moderate, low, and very low.[10] It considers “within-study risk of bias (methodological quality), directness of evidence, heterogeneity, precision of effect estimates, and risk of publication bias.”[10] A study was defined as being “high” quality if it was a randomized trial. Quality ratings decreased with several factors including being observational study design, risk of bias, and small sample size.

The risk of bias assessment was completed using Covidence which is integrated with the Cochrane Collaboration's tool for assessing risk of bias.[10] Each study was assigned a rating of low, high, or unclear risk of bias for each assessment category. Both the GRADE and risk of bias scores were independently assessed by two authors (G.N and J.P.). Contradicting scores were mediated through discussion.

Search Results

A total of 575 studies were identified across all databases (Ovid MEDLINE, Cochrane CENTRAL, and EMBASE). Covidence identified 137 duplicates, leaving 438 records to screen. A total of 327 studies were excluded in the title and abstract screening and another 105 studies were excluded in the full-text screening. The most common reasons for exclusion were studies that utilized simulation training as the primary intervention or studies not specific to ophthalmology. The screening processes, as well as exclusion criteria, are illustrated in [Fig. 1]. Thirteen studies were reviewed for this meta-analysis.

Fig. 1 Literature search flow diagram: surgical skills educational interventions to improve surgical performance outcomes in medical students, ophthalmology residents, and ophthalmology trainees.

Quality of Studies

The majority of the studies received a GRADE Score of “low” due to being nonrandomized observational or retrospective cohort studies. Two randomized controlled trials received a score of “high”[11] [12] (see [Table 1] for complete GRADE scores and reasoning).

Table 1 Grades of recommendation, assessment, development, and evaluation (GRADE) scores for all included studies

Study (year)	Grade score	Reason
Ezra et al[13] (2009)	Low	No method of randomization to recruit residents was stated. Residents were compared with themselves (pre-vs. postintervention)
Rogers et al[14] (2009)	Low	Nonrandomized study, retrospective cohort study
Yulan et al[15] (2013)	Low	Nonrandomized study, retrospective cohort study
Kloek et al[3] (2014)	Low	Nonrandomized study, retrospective cohort study
Kaplowitz et al[16] (2016)	Low	Nonrandomized study, retrospective cohort study
Gershuni et al[12] (2013)	High	Randomized controlled trial with no significant sources of bias
Shen et al[17] (2015)	Low	Nonrandomized prospective study, some outcomes were based on subjective measures
Yu et al[18] (2016)	Low	Nonrandomized observational study
Vagge et al[19] (2017)	Low	Nonrandomized prospective cohort study, some outcomes were based on subjective measures
Borboli-Gerogiannis et al[20] (2019)	Low	Nonrandomized study with possible bias in technology difference between 2004 and 2015
Mishra et al[11] (2017)	High	Randomized controlled trial with no significant sources of bias
Ramani et al[1] (2018)	Low	Nonrandomized study, retrospective cohort study
Pantanelli et al[21] (2018)	Low	Nonrandomized study, retrospective cohort study

Risk of Bias

There were no studies scored with a low risk of bias across all bias assessment categories. Mishra et al[11] was found to have the least amount of bias overall, with six of seven domains being in the low risk category. Blinding of participants and personnel for all outcomes was the lowest scoring assessment category, with all 13 studies judged with high risk of bias. This was in large part due to the majority of the studies being nonrandomized trials or not describing any blinding method used. Selective outcome reporting was the best domain with all 13 studies judged with low risk of bias. The studies included all data points and were analyzed with appropriate statistical tests. [Fig. 2] summarizes the risk of bias assessment for all the included studies.

Fig. 2 Risk of bias summary: a summary of the author's judgements on each risk of bias item for each included study. Green circle = low risk of bias; yellow circle = unclear risk of bias; red circle = high risk of bias. Software used: Review Manager v5.4 (The Nordic Cochrane Centre, The Cochrane Collaboration, Copenhagen Denmark).

Data Synthesis and Analysis

Summary tables showing study, population characteristics, interventions, and outcomes were created for all key questions. Meta-analysis assessed several surgical performance outcomes: time for completion of surgical task, level of preparedness as assessed by faculty, competency score, and number of surgeries with complications. Interventions were interpreted as having a positive effect on surgical performance outcomes if their results showed any of the following: decreased time for surgical task completion, increased level of preparedness, increased competency score, or decreased surgical complications. Six studies reported a continuous measure (time for completion of surgical task and competency score) and seven studies reported a discrete outcome (number of complications) or ordinal outcome (level of preparedness). Continuous measures were converted to Hedges' g values as standardized mean differences (SMDs), while discrete and ordinal outcomes were converted to odds ratios (ORs) based on calculations from study-reported numbers of persons with and without the event for each group. Some studies used scales that increased with positive outcomes while others decreased. For differences in the direction of the scales, the mean values from one set of studies were multiplied by −1 or subtracted from the maximum possible value for the scale.

Random-effects meta-analyses on SMDs and ORs were conducted for five educational interventions as follows: (1) didactic lectures, (2) video-based education, (30 wet-laboratory curriculum, (4) direct supervision and feedback from an attending surgeon, and (5) stepwise method. Discrete and ordinal measures were selected over continuous measures and outcomes assessing time for completion of a surgical task over competency score were selected when multiple outcomes were reported for the same study. The DerSimonian and Laird (inverse variance) model was used to calculate the pooled effect estimate with a 95% confidence interval (CI) across studies. RevMan version 5.4 was used for all analyses.

Results

A total of 575 studies were reviewed. After rigorous screening, 13 studies (n = 8790 surgical cases; n = 115 trainees) were identified that met inclusion criteria for this meta-analysis, including 2 randomized control trials (RCTs) and 11 non-RCTs.[1] [3] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] The range of mean ages of participants across all trials was between 25 and 37 years, although one trial excluded preresidency trainees and therefore had a slightly older population (range: 28–46 years).[18] All included studies are listed in [Table 2] with a summary of study characteristics in [Table 3].

Table 2 Study characteristics of all included studies of surgical skills educational interventions to improve surgical performance outcomes in medical students, ophthalmology residents, and ophthalmology trainees (13 studies)

Study	Quality	Country	No. of participants (preintervention/postintervention)	No. of surgical cases (preintervention/postintervention)	Brief population description	Brief description of curriculum/program interventions	Outcomes measured
Borboli-Gerogiannis et al[20] (2019)	Low	The United States	Not provided	804/1,282	Residents from the 2004–2005 and 2014–2015 class	Cataract surgery training curriculum Pre-OR: wet laboratory, faculty supervision, didactic lectures Intra-OR: faculty supervision, stepwise method Post-OR: faculty feedback (verbal/written)	Incidence of intraoperative complications
Ezra et al[13] (2009)	Low	The United Kingdom	14/14 (same group)	N/A	Residents Mean age: 32.2 years (SD = 2.8)	1-day microsurgical skills course 5 didactic lectures (30 minutes each), wet laboratory (240 minutes)	Time to complete microsurgical task, OSATS score for microsurgical task
Gershuni et al[12] (2013)	High	The United States	10/11	N/A	MS4 students entering a surgery specialty Mean age: preintervention = 25.9 years (SD = 1.1), postintervention = 26.2 years (SD = 2.1)	Proficiency-based suturing and knot-tying curriculum: didactic lecture, suturing training session (3 hours), suturing/knot-tying video tutorials	Time to complete suturing task, technical proficiency scores for suturing task
Kaplowitz et al[16] (2016)	Low	The United States	4/4 (different groups)	223/242	PGY-4 residents from the 2008–2009 and 2011–2012 class	Stepwise biometry lecture series curriculum: didactic lectures, practical session, direct supervision and feedback from an attending	Percent of cases that finished within a range of Diopters of the target refraction
Kloek et al[3] (2014)	Low	The United States	7/16	N/A	PGY-3 and PGY-4 residents from the 2010–2012 classes	Stepwise introduction to cataract surgery curriculum: stepwise method with deliberate practice, supervision of an attending	Resident self-assessment of confidence Faculty assessment of resident preparedness
Mishra et al[11] (2017)	High	The United States	7/4	N/A	Residents from all post graduate years who had no prior experience in lid laceration or blepharoplasty procedures	5-day surgical wet laboratory video curriculum: surgical video tutorial and text instructions for procedure, wet laboratory, didactic lectures	Time to complete lid laceration and blepharoplasty procedures Esthetic grading of procedures
Pantanelli et al[21] (2018)	Low	The United States	12/9	1,659/2,044	PGY-4 residents from 2011–2018	6-week cataract surgery training curriculum: didactic lectures (1-hour/week), stepwise method, wet laboratory, video review, formative feedback from faculty	Incidence of posterior capsule tears
Ramani et al[1] (2018)	Low	India	Not provided	232/232	1st, 2nd, and 3rd year residents 2012–2014	2-month cataract surgery training curriculum: wet laboratory, stepwise method	Incidence of intraoperative and postoperative complications
Rogers et al[14] (2009)	Low	The United States	Not provided	823/1,009	PGY-3 residents from 1998–2008	Cataract surgery training curriculum First year: wet laboratory Second year: formative feedback from faculty, deliberate practice	Incidence of posterior capsule tears or vitreous loss
Shen et al[17] (2015)	Low	The United States	8/8 (different groups)	N/A	PGY-4 residents from 2012–2014	Glaucoma surgical curriculum Pre-OR: video tutorials, proctored and independent surgical skills practice Intra-OR: direct supervision from attending Post-OR: structured feedback from attending, video review, independent practice	GRASIS rating from chief residents
Vagge et al[19] (2017)	Low	The United States	12/12 (same group)	N/A	PGY-1 and PGY-2 residents Mean age: 28.3 years (SD = 1.5), range: 27–32 years	2-hour strabismus surgery suture course: didactic lecture (50 minutes), wet laboratory (70 minutes)	International Council of Ophthalmology – Ophthalmology Surgical Competency Assessment Rubrics (ICO-OSCAR) score for strabismus surgery Resident self-assessment of confidence
Yu et al[18] (2016)	Low	China	12/12 (same group)	N/A	Ophthalmology trainees who graduated from a residency training program Mean age: 37 years (SD = 5), range: 28–46 years	3-month cataract surgery training program: wet laboratory, stepwise method, deliberate practice, video review, direct supervision from a trainer, verbal and diagram-assisted feedback	International Council of Ophthalmology – Ophthalmology Surgical Competency Assessment Rubrics (ICO-OSCAR) score for phacoemulsification
Yulan et al[15] (2013)	Low	China	4/4 (same group)	120/120	Residents without previous extracapsular cataract extraction experience from March 2009 to September 2011	Step-by-step phacoemulsification training program: study of basic literature, wet laboratory, stepwise method, direct supervision from an attending, surgical video discussions	Completion rate and main complications in phacoemulsification procedures

Abbreviations: N/A, not available; PGY, postgraduate year; SD, standard deviation.

Table 3 Summary of study characteristics of all included studies of surgical skills educational interventions to improve surgical performance outcomes in medical students, ophthalmology residents, and ophthalmology trainees (13 studies, n = 8,790 surgical cases, and n = 115 trainees)

Characteristics	No. of studies	%
All studies	13	100
Study design
RCT	2	15
RCS	9	69
PCS	2	15
Quality rating
High	2	15
Low	11	85
Conducted in the United States	9	69
Intervention type
Didactic lectures	7	54
Video-based education	5	38
Wet laboratory	9	69
Feedback/supervision	8	62
Stepwise method	7	54
Primary intervention outcomes
Complication rate	6	46
Competency score	6	46
Surgical task completion time	3	23
Level of preparedness	1	8

Abbreviations: PCS, prospective cohort study; RCS, retrospective cohort study; RCT, randomized controlled trial.

Analysis of Educational Interventions

Key question 1: Does teaching through didactic lectures improve surgical performance outcomes?

Didactic lectures incorporated into surgical curriculums involve teaching the steps of the surgical task, drawing out the relevant anatomy for each surgery, and teaching appropriate surgical techniques for instrument handling. Four studies were identified that evaluate the efficacy of didactic lectures on surgical performance. Two studies reported the effect of their interventions on the time of completion of a surgical task.[12] [13] The other two studies measured changes in surgical competency scores.[11] [19] The effects of the didactic lectures intervention on the outcomes were very similar across all compared studies. The pooled SMD was 1.04 (95% CI: 0.51–1.57; four effects [four studies]; n = 65; I ² = 0%; [Table 4]). Our findings suggest didactic surgical teaching has a positive effect on surgical performance, resulting in a decrease in time to task completion and an increase in surgical competency scores.

Table 4 Summary of meta-analysis results for surgical performance outcomes for various educational interventions

	No. of studies	Type of effect	Pooled result (95% CI)	No. of studies (no. of effects)	I 2 (%)	Tau2	n	Range of effects	Effects, median (IQR)
Didactic lectures outcomes	4	SMD	1.04 (0.51, 1.57)	4 (4)	0	0	65	0.51–1.34	1.05 (0.755, 1.22)
Video-based education outcomes	4	SMD	2.49 (0.36, 4.63)	4 (4)	90	4.04	69	0.51–9.66	1.09 (0.675, 5.5)
Wet laboratory curriculum outcomes	5	OR	3.10 (1.95, 4.93)	5 (5)	67	0.17	8,325	1.97–7.17	2.79 (2.105, 6.52)
Direct supervision and feedback from an attending surgeon outcomes	5	OR	3.49 (2.17, 5.60)	5 (5)	68	0.18	7,485	1.97–7.86	4.62 (2.38, 6.865)
Stepwise method outcomes	6	OR	3.84 (2.66, 5.55)	6 (6)	39	0.08	6,968	2.24–7.86	5.245 (2.79, 7.17)

Abbreviations: CI, confidence interval; IQR, interquartile range; OR, odds ratio; SMD, standardized mean difference (Hedges' g).

Key question 2: Does the use of video-based education improve surgical performance outcomes?

Video-based education was defined as the utilization of either video tutorials presurgery or video review postsurgery as part of a surgical curriculum. Three studies reported the effect of video-based education on surgical competency,[11] [17] [18] while one study measured the time of completion of a surgical task.[12] The effects of the video-based education interventions varied substantially, and the result of the pooled analysis is likely to be affected by the high degree of heterogeneity between studies (pooled SMD = 2.49 [95% CI: 0.36–4.63]; four effects [four studies]; n = 69; I ² = 90%; [Table 4]). The largest effect across all continuous outcomes was found with a cataract surgery training program. Ophthalmology trainees were required to review their surgery videos with an attending during weekly discussion meetings.[18] At 3-month follow-up, the mean competency score for trainees had improved from 2.3 (SD = 0.2) to 4.3 (SD = 0.2) which converted into an SMD of 9.66 (95% CI: 6.56–12.75; [Fig. 3]). Our findings indicate that video-based education has a positive effect on surgical performance outcomes resulting in improvement in competency scores and decreased time to task completion.

Fig. 3 Surgical skills interventions outcomes summary: standardized mean difference and odds ratio between postintervention and preintervention groups, by intervention. CI, confidence interval.

Key question 3: Does a structured wet-laboratory curriculum improve surgical performance outcomes?

Wet-laboratory curriculums described in the studies include training sessions with either goat or porcine eyes to practice microsurgical skills. Written precognitive and postcognitive tests were utilized to assess resident knowledge of the instruments, anatomy, and steps involved with cataract surgery in one study.[14] [22] Five studies were identified that evaluated the impact of surgical wet-laboratory curricula on ophthalmic surgical performance. All five studies reported the effect of their interventions on the number of surgical complications.[1] [14] [15] [20] [21] More specifically, the number of posterior capsule tears in cataract surgery was compared between pre- and postintervention groups. The effects of wet-laboratory curriculum intervention varied between studies. The pooled OR was 3.10 (95% CI: 1.95–4.93; 5 effects [five studies]; n = 8,325; I ² = 67%; [Table 4]). The studies reported an improvement in surgical performance and a reduction in surgical complications using a structured surgical wet-laboratory curriculum.

Key question 4: Does direct supervision and feedback from an attending surgeon improve surgical performance outcomes?

Five studies were identified that evaluated the impact of direct supervision and feedback from an attending surgeon on surgical performance in surgical trainees. Direct supervision and feedback were incorporated into the various surgical curriculums by having attending surgeons oversee wet-laboratory practice sessions and intraoperative surgeries. Faculty provided verbal real-time and written delayed feedback. Two studies emphasized the use of formative feedback which focuses on giving participants the resources they need to improve and identify their strengths and weaknesses.[14] [21] This differs from the more traditional summative feedback which evaluates only proficiency. Four studies reported the effect of attending feedback on the number of surgical complications,[14] [16] [20] [21] while one study evaluated the level of preparedness of residents as judged by faculty.[3] The pooled OR was 3.49 (95% CI: 2.17–5.60; five effects [five studies]; n = 7,485; I ² = 68%; [Table 4]), suggesting that the use of direct supervision and feedback from an attending surgeon has a positive effect on surgical performance outcomes. All five studies reported an increase in surgical performance, a reduction in surgical complications, and an improvement in resident preparedness using direct supervision and feedback from an attending surgeon.

Key question 5: Does implementing a stepwise method of teaching improve surgical performance outcomes?

The stepwise method of surgical teaching involves deconstructing surgical procedures into specific critical steps. The learner practices many repetitions of each step before performing the entire surgical procedure. Five studies reported the effect of utilizing the stepwise teaching method on the rate of complications in cataract surgery.[1] [15] [16] [20] [21] One study evaluated the level of preparedness of residents as judged by faculty.[3] This intervention had the greatest effect of the dichotomous outcomes with a pooled OR of 3.84 (95% CI: 2.66–5.55; six effects [six studies]; n = 6,968; I ² = 39%; [Table 4]). These studies reported an increase in surgical performance using the stepwise method of teaching with a reduction in surgical complications and an increase in level of preparedness.

Discussion

Our meta-analysis included 13 studies, evaluating the efficacy of following five methods of surgical education: (1) didactic lectures, (2) surgical videos, (3) surgical wet laboratories, (4) stepwise method, and (5) direct supervision and feedback. All five interventions were found to be effective and consistently beneficial.

Among the interventions that reported a continuous outcome measure, the video-based education interventions were more effective than didactic lecture interventions. Video review is effective for surgical teaching, as it allows participants to view and repeat at their own paces and increases accessibility for faculty to identify areas of improvement.[21] The cataract surgery training program reported by Yu et al showed a more substantial effect compared with the other three video-based education studies. This was likely explained by differences in curriculum structure and guidelines, including a longer training time and rigorous protocol. The training phase for this study was 3 months compared with other studies which ranged from 1 day to 7 weeks.[11] [12] [17] [18] Additionally, the program expectations were rigorous. Trainees were taught one step of cataract surgery at a time and were required to meet a certain International Council of Ophthalmology – Ophthalmology Surgical Competency Assessment Rubrics (ICO-OSCAR) score before being able to proceed to the next surgical step.[18] In comparison, other studies had more flexible curriculums, allowing for self-study or independently practice with no mandatory goals.[11] [12] [17]

When comparing the interventions that reported a binary outcome, wet laboratory, direct supervision and feedback, and stepwise method, all had extremely favorable results with an OR > 3. Implementation of a stepwise method of teaching, yielded the most positive relationship with surgical performance.

Wet-laboratory curriculums are outstanding for introducing microsurgical techniques as they allow participants to orient themselves to using their fine stereoscopic vision, improve dexterity,[21] and grow surgical confidence without excessive worry of patient harm.

Direct supervision and feedback from attending surgeons highlight a personalized approach to teaching and can be easily integrated into any curriculum. Participants may establish specific goals by having an experienced mentor who follows a trainee's progress longitudinally.

The stepwise method of teaching emphasizes mastery of surgical fundamentals. Participants were required to meet a certain level of proficiency with each step of the surgery before progressing which may have indirectly increased the amount of time they spent practicing.[3] This would mean that trainees were more prepared when finally performing the entire surgery which may have contributed to the lower number of surgical complications observed compared with the other interventions.

When the studies were stratified by level of training of their candidates (medical students of postgraduate year (PGY)-1/PGY-2/PGY-3, PGY-4, and attending physician), there were noted differences in the effectiveness of the methods for different groups. The stepwise teaching method, wet-laboratory curriculum, and direct supervision and feedback interventions were most effective for PGY-4 level residents. Video based education was most effective for ophthalmologists at the attending physician level and didactic lectures did not show a significant difference between the groups. We believe that that these teaching methods generally had a larger impact in more experienced candidates due to their increased level of exposure to microsurgical procedures. Compared with participants who have just started training, PGY-4 and attending level candidates have the opportunity to immediately demonstrate improved surgical outcomes due to the higher volume of procedures they participate in. They have a stronger foundation and thus can focus more on refining rather than learning the skills targeted in the curriculums.

To compare pooled effects between SMDs and ORs, a transformation formula was used as described in the Cochrane handbook.[10] Across all studies, video-based education interventions had the largest effect. Video-based education combines both visual and reflective learning, two effective strategies that are crucial for improving proficiency with complex skills. The results of this meta-analysis can give institutions an overview of which interventions to implement when forming a microsurgical skills curriculum. One phacoemulsification training program incorporated both video-based education and a stepwise method of teaching into its study. The result of this study was an OR of 7.17 (95% CI: 1.58–32.50; n = 240; [Fig. 3]), the highest effect out of studies with a n > 100 sample size.[15]

Some of the programs implemented in the studies were short term, occurring within a day, while others were longer term, occurring over a span of years. We can see how both time frames offer unique benefits for ophthalmology microsurgery. Short-term curriculums are easier to administer, as they require less resources and time investment. This allows them to be more applicable across a wider range of institutions. They also can be more focused and target a certain skill. Since microsurgery is known to have a steep learning curve, the ability of short-term programs to target weaknesses is valuable to rapidly improve surgical outcomes. Long-term curriculums can integrate multiple teaching strategies that can complement one another, as seen in programs that incorporate stepwise teaching, wet laboratories, and formative feedback. Being able to track the progress of trainees over the course of years allows for more comprehensive assessments. Additionally, these longer curriculums allow for superior skill retention compared with shorter programs due to the continued follow-up that occurs.

While our study did not include surgical training programs outside of ophthalmology, a review of the literature shows that similar to the findings of our meta-analysis, stepwise approach or surgical video review were found to be effective surgical training methods in general surgery and urology.[23] [24] [25]

Curriculums focusing on cataract surgical simulation platforms (such as EYESI surgical simulator) as the primary intervention were specifically excluded from this study for two reasons. First, the benefits and utility of this surgical education intervention have been previously published and well established.[7] [8] [9] Second, due to cost and availability, these tools are not available for all surgical training programs. We therefore focused this meta-analysis on interventions that are broadly available and applicable.

Limitations

There were some limitations associated with this review. Studies which did not design their interventions into a program or curriculum framework were excluded, even if their trials addressed improvements in surgical performance outcomes. Second, interventions implemented in fields other than ophthalmology were excluded, although some did utilize the same educational methods explored in included studies. Third, since the study population included ophthalmology residents, medical students, and other ophthalmology trainees, the participant's prior surgical experience may be a confounding variable that contributed to results. Three included studies had active surgical simulators as part of their program.[3] [14] [20] Although these simulators were not the primary intervention of the curriculum and their effects were not specifically discussed, their presence could have influenced the benefit of other training modalities. Limitations of our meta-analysis include sample size, retrospective study designs, heterogeneity, and subjective nature of assessments. Small sample sizes were likely due to the size of resident classes. Randomized trials on educational interventions are difficult to conduct, as they deny participants in the control group access to potentially helpful resources,[1] thus leading to the retrospective design of most studies. Surgical performance outcomes were heterogeneous, with three out of five outcome groups having an I ² > 60% ([Table 4]). This suggests that the studies varied substantially, making it difficult to draw conclusions. Finally, outcomes such as competency score and perceived resident preparedness are subjective.

For future studies, randomized controlled trials focusing on the effects of each intervention separately on objective measures would be valuable. The issue of withholding advantageous interventions may be addressed by using medical student volunteers or allowing all residents access to the resources after conclusion of data collection or with a cross-over study design.

Conclusion

The following five interventions evaluated in this paper were found to be effective methods of improving performance outcomes in ophthalmic microsurgery: (1) didactic lectures, (2) video-based education, (3) surgical wet laboratory, (4) stepwise method, and (5) direct supervision and feedback. Our meta-analysis concludes that video-based education and stepwise teaching interventions are the most effective methods for teaching a microsurgery training curriculum. Combining the strengths in these methods should be considered when implementing and adjusting ophthalmic surgical skills curriculums.

Conflict of Interest

None declared.

Acknowledgments

We acknowledge the support of the University of Maryland, Baltimore, Institute for Clinical and Translational Research (ICTR) and the National Center for Advancing Translational Sciences (NCATS) Clinical Translational Science Award (CTSA; grant number: 1UL1TR003098).

• References

• 1 Ramani S, Pradeep TG, Sundaresh DD. Effect of wet-laboratory training on resident performed manual small-incision cataract surgery. Indian J Ophthalmol 2018; 66 (06) 793-797
• 2 Pujari A, Basheer S, Rakheja V. et al. Extraocular muscle surgery on goats' eye: An inexpensive technique to enhance residents' surgical skills. Indian J Ophthalmol 2019; 67 (10) 1688-1689
• 3 Kloek CE, Borboli-Gerogiannis S, Chang K. et al. A broadly applicable surgical teaching method: evaluation of a stepwise introduction to cataract surgery. J Surg Educ 2014; 71 (02) 169-175
• 4 Connolly A, Hansen D, Schuler K, Galvin SL, Wolfe H. Immediate surgical skills feedback in the operating room using “SurF” cards. J Grad Med Educ 2014; 6 (04) 774-778
• 5 Baker RC, Spence RA, Boohan M. et al. A novel approach to improve undergraduate surgical teaching. Ulster Med J 2015; 84 (01) 30-36
• 6 Lorch AC, Kloek CE. An evidence-based approach to surgical teaching in ophthalmology. Surv Ophthalmol 2017; 62 (03) 371-377
• 7 Rasmussen RC, Grauslund J, Vergmann AS. Simulation training in vitreoretinal surgery: a systematic review. BMC Ophthalmol 2019; 19 (01) 90
• 8 Bergqvist J, Person A, Vestergaard A, Grauslund J. Establishment of a validated training programme on the EYESI cataract simulator. A prospective randomized study. Acta Ophthalmol 2014; 92 (07) 629-634
• 9 Vergmann AS, Vestergaard AH, Grauslund J. Virtual vitreoretinal surgery: validation of a training programme. Acta Ophthalmol 2017; 95 (01) 60-65
• 10 Higgins J, Thomas J, Chandler J. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.0. The Cochrane Collaboration; John Wiley & Sons: 2019
• 11 Mishra K, Mathai M, Della Rocca RC, Reddy HS. Improving resident performance in oculoplastic surgery: a new curriculum using surgical wet laboratory videos. J Surg Educ 2017; 74 (05) 837-842
• 12 Gershuni V, Woodhouse J, Brunt LM. Retention of suturing and knot-tying skills in senior medical students after proficiency-based training: results of a prospective, randomized trial. Surgery 2013; 154 (04) 823-829 , discussion 829–830
• 13 Ezra DG, Aggarwal R, Michaelides M. et al. Skills acquisition and assessment after a microsurgical skills course for ophthalmology residents. Ophthalmology 2009; 116 (02) 257-262
• 14 Rogers GM, Oetting TA, Lee AG. et al. Impact of a structured surgical curriculum on ophthalmic resident cataract surgery complication rates. J Cataract Refract Surg 2009; 35 (11) 1956-1960
• 15 Yulan W, Yaohua S, Jinhua T, Min W. Step-by-step phacoemulsification training program for ophthalmology residents. Indian J Ophthalmol 2013; 61 (11) 659-662
• 16 Kaplowitz K, Hong BY, Chou TY, Abazari A, Honkanen R. Improved refractive outcomes of postgraduate year 4 cataract surgery after implementing a stepwise biometry lecture series reinforced by self-assessment at a teaching program. J Cataract Refract Surg 2016; 42 (04) 524-529
• 17 Shen LQ, Kloek CE, Turalba AV. Assessing the effect of a glaucoma surgical curriculum in resident physicians. JAMA Ophthalmol 2015; 133 (09) 1077-1080
• 18 Yu AY, Wang QM, Li J, Huang F, Golnik K. A cataract surgery training program: 2-year outcome after launching. J Surg Educ 2016; 73 (05) 761-767
• 19 Vagge A, Gunton K, Schnall B. Impact of a strabismus surgery suture course for first- and second-year ophthalmology residents. J Pediatr Ophthalmol Strabismus 2017; 54 (06) 339-345
• 20 Borboli-Gerogiannis S, Jeng-Miller KW, Koulisis N. et al. A comprehensive surgical curriculum reduced intra-operative complication rates of resident-performed cataract surgeries. J Surg Educ 2019; 76 (01) 150-157
• 21 Pantanelli SM, Papachristou G, Callahan C, Chen M, Khalifa Y. Wet lab-based cataract surgery training curriculum for the PGY 2/PGY 3 ophthalmology resident. MedEdPORTAL 2018; 14: 10782
• 22 Lee AG, Greenlee E, Oetting TA. et al. The Iowa ophthalmology wet laboratory curriculum for teaching and assessing cataract surgical competency. Ophthalmology 2007; 114 (07) e21-e26
• 23 van der Poel MJ, Huisman F, Busch OR. et al. Stepwise introduction of laparoscopic liver surgery: validation of guideline recommendations. HPB (Oxford) 2017; 19 (10) 894-900
• 24 Augestad KM, Butt K, Ignjatovic D, Keller DS, Kiran R. Video-based coaching in surgical education: a systematic review and meta-analysis. Surg Endosc 2020; 34 (02) 521-535
• 25 Walsh PC, Marschke P, Ricker D, Burnett AL. Use of intraoperative video documentation to improve sexual function after radical retropubic prostatectomy. Urology 2000; 55 (01) 62-67

Publication History

Received: 27 November 2020

Accepted: 19 August 2021

Article published online:
05 December 2021