Keywords
resident education - cataract surgery - video coaching - curriculum - surgical curriculum
Medical and surgical training has long followed an apprenticeship model with the historical teaching of “See one, do one, teach one.”[1]
[2] As medicine has advanced, both technologically and in the ability to measure patient outcomes, there has been a push toward competency-based assessment. Starting in 1998, the American Council on Graduate Medical Education (ACGME) developed six areas of competency that residents had to meet: patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism, and systems-based practice. The ACGME has also recently instituted the CLER (Clinical Learning Environment Review) Pathways to Excellence. Rather than simply counting cases, monitoring training outcomes became required and expectations on topics such as patient safety, health care quality, and supervision were increased.[3]
[4]
[5]
[6]
More than 3 million cataract surgeries are performed in the United States annually[7] and cataract surgery is the most commonly performed intraocular surgery by residents during residency training. Currently, the minimum number of cataract surgeries required by the ACGME is 86 with most programs easily exceeding this number.[8] After the ACGME mandate, there were many studies performed to evaluate resident outcomes, methods of teaching cataract surgery, and the cost of resident educational modalities.[3]
[9]
[10]
[11]
[12]
[13] There was a demonstrable learning curve for phacoemulsification cataract surgery with reductions in complications and improvements in efficiency with more experience.[14] Newer methodologies of training and learning include wet laboratory curricula,[9]
[12] virtual reality training,[15]
[16] surgical simulators such as EyeSi, and tools to evaluate progression,[3]
[10]
[11] all with the goal of having surgeons master the learning curve earlier with fewer complications. When discussing these new modalities, cost has been an increasingly important consideration to ensure ability for implementation, sustainability, and positive net benefit.[13]
One area of training that many other surgical fields, including general and orthopedic surgery, have adapted is personalized video-based feedback to improve performance in surgeries.[17]
[18] Reflective surgical practice has long been effective in enhancing education as it has been used at conferences, presentations, and research forums.[19] Furthermore, video review often allows root cause analysis to learn from habits or mistakes that lead to system errors.[20] In a recent general surgery study, comprehensive surgical coaching enhanced the surgery training and led to superior skill acquisition when compared with conventional training.[21]
Learning through simulators, operating room experience, and other methods may be enhanced by a formal curriculum, including objective assessment of actual surgical cases, debriefing, feedback from multiple levels of surgeons, and self-reflection. The aim of this study was to develop, implement, and evaluate a video-coaching curriculum for cataract surgery in a U.S. residency program.
Methods
A group of experienced faculty members met to develop a 6-month pilot video-coaching curriculum for the Ophthalmology Residency Program at Emory University in Atlanta, Georgia.
The developed curriculum included bi-weekly video conferences running from January 2016 through June 2016. At each session, one or two senior residents showed a surgical video he/she performed with a faculty surgeon demonstrating a specific topic. The cases were chosen by the resident in consultation with the faculty surgeon. Topics were focused on either an intraoperative complication or a complex case requiring advanced techniques. The sessions were 1 hour long and performed in an open format with the video running at one to two times the real speed. The video was projected on a 10 × 8 ft screen so that all participants could view the surgery footage. The resident and faculty physician went through the case in a step-by-step fashion with frequent pauses for discussion. Attendees included faculty physicians and individuals in all levels of training such as resident physicians (PGY2–4) and medical students. The faculty mentor would ensure that all important aspects of the case were reviewed.
In addition to the resident physician describing his/her thought process on certain steps, the faculty physician provided tips and pitfalls to avoid. Frequent questions were asked of the residents and students in the audience with input from all faculty physicians. The environment was designed to be collaborative, nonjudgmental, and educational.
Each operating resident, faculty attending surgeon, and observing resident filled out a post-session questionnaire ([supplementary Figs. S1] and [S2]). The questionnaires included questions on video coaching, trainee's performance, mentor's style, reasons for stalls, and an Objective Structured Assessment of Technical Skill (OSATS) evaluation of the surgery. Likert scales were used to provide quantitative data as answers for the majority of the questions with a few questions permitting qualitative feedback. The questionnaire responses were not made available to the presenting resident, the audience, or the attending physicians and the results had no bearing on judging the resident's surgical proficiency. This was made clear to all in attendance at the beginning of each session.
The questionnaire response data were entered into an Excel spreadsheet and statistical analyses were performed between observing resident, presenting resident, and faculty attendee questionnaires. Analyses included mean and standard deviation comparisons, correlation coefficients, and Student's t-tests.
Results
Through the 6-month pilot study, 6 senior resident physicians presented a total of 24 cases during 12 sessions. There were 13 presenting resident questionnaire responses, 99 observing resident responses, 2 medical student responses, and 35 faculty responses after the 12 sessions. There were four faculty members with at least one of the faculty present at each session to provide faculty responses.
[Table 1] illustrates the OSATS scores, including scores for each individual category and the overall average score for each participating group. Presenting residents have a lower average score of 3.32 when compared with both observing residents' score of 4.14 (p < 0.001) and faculty physicians' score of 4.20 (p < 0.001). There is no significant difference between observing resident and faculty scores.
Table 1
Post-session objective structured assessment of technical skill (OSATS) scores
|
Respect for tissue
|
Time and motion
|
Knowledge and handling of instrument
|
Flow of operation
|
Use of assistants
|
Knowledge of specific procedure
|
Average score
|
|
Presenting residents
|
3.31
|
3.00
|
3.54
|
3.23
|
3.54
|
3.31
|
3.32
|
|
Observing residents
|
4.09
|
4.04
|
4.26
|
4.08
|
4.18
|
4.18
|
4.14
|
|
Faculty
|
4.19
|
4.09
|
4.26
|
4.13
|
4.28
|
4.27
|
4.20
|
Notes: p-Values are <0.001 for presenting residents versus observing residents and presenting residents versus faculty. p-Value is 0.18 and is not statistically significant for observing residents versus faculty.
The perceived benefits of the video-coaching curriculum are demonstrated in [Table 2]: 100% of residents and faculty found the curriculum beneficial. The majority of participants in each category found benefit in recall of technical cues (11 of 13 presenting residents, 76 of 99 observing residents, 29 of 35 faculty), avoiding errors (12 of 13 presenting residents, 81 of 99 observing residents, 31 of 35 faculty), flow of performance (9 of 13 presenting residents, 50 of 99 observing residents, 27 of 35 faculty), recall of procedural steps (8 of 13 presenting residents, 56 of 99 observing residents, 26 of 35 faculty), and overall performance (12 of 13 presenting residents, 66 of 99 observing residents, 31 of 35 faculty). The majority of presenting residents and faculty found benefit in time of task (7 of 13 and 28 of 35, respectively), but observing residents (36 of 99) did not perceive this as a benefit.
Table 2
Benefits of video coaching
|
Presenting residents (13)
|
Observing residents (99)
|
Faculty (35)
|
|
Overall benefit
|
13 (100%)
|
99 (100%)
|
35 (100%)
|
|
Recall of technical cues
|
11 (85%)
|
76 (77%)
|
29 (83%)
|
|
Time of task
|
7 (53%)
|
36 (36%)
|
28 (80%)
|
|
Avoiding errors
|
12 (92%)
|
81 (82%)
|
31 (89%)
|
|
Flow of performance
|
9 (69%)
|
50 (51%)
|
27 (77%)
|
|
Recall of procedural steps
|
8 (62%)
|
56 (57%)
|
26 (74%)
|
|
Overall performance
|
12 (100%)
|
66 (67%)
|
31 (89%)
|
[Table 3] illustrates the participants' overall performance rating and the resulting statistical analysis. Observing residents and faculty had similar average score (1.7 and 1.6, respectively) with the majority choosing excellent (54 of 99 and 20 of 35, respectively), while presenting residents overwhelmingly chose satisfactory (8 of 13) with a statistically significant (p < 0.001) lower average score (2.5).
Table 3
Post-session overall performance
|
Presenting residents (13)
|
Observing residents (99)
|
Faculty (35)
|
|
Excellent (1)
|
2 (15%)
|
54 (55%)
|
20 (57%)
|
|
Average (2)
|
2 (15%)
|
22 (22%)
|
12 (34%)
|
|
Satisfactory (3)
|
8 (62%)
|
22 (22%)
|
0 (0%)
|
|
Unsatisfactory (4)
|
1 (8%)
|
1 (1%)
|
3 (8.6%)
|
|
Average 4-point score
|
2.5
|
1.7
|
1.6
|
Notes: p-Values are <0.001 for presenting residents versus observing residents and presenting residents versus faculty. p-Value is 0.28 and is not statistically significant for observing residents versus faculty.
[Table 4] illustrates the comfort level of the various groups with residents performing different steps of cataract surgery: 100% of presenting residents (13 of 13) felt comfortable with each individual step of the surgical procedure, except nucleus disassembly where 85% (11 of 13) noted comfort. Attending physicians had varying comfort levels with 86% (30 of 35) expressing confidence for nucleus disassembly, quadrant removal, and cortical clean-up, while 77% (27 of 35) had comfort with closure, 89% (31 of 35) had comfort with intraocular lens insertion, 91% (32 of 35) had comfort with capsulorrhexis, and 100% (35 of 35) had comfort with incisions.
Table 4
Comfort with resident performing task autonomously
|
Presenting residents (13)
|
Observing residents (99)
|
Faculty (35)
|
|
Incisions
|
13 (100%)
|
75 (76%)
|
35 (100%)
|
|
Capsulorrhexis
|
13 (100%)
|
53 (54%)
|
32 (91%)
|
|
Nucleus disassembly
|
11 (85%)
|
49 (49%)
|
30 (86%)
|
|
Quadrant removal
|
13 (100%)
|
45 (45%)
|
30 (86%)
|
|
Cortical clean up
|
13 (100%)
|
46 (46%)
|
30 (86%)
|
|
IOL insertion
|
13 (100%)
|
70 (71%)
|
31 (89%)
|
|
Closure
|
13 (100%)
|
65 (66%)
|
27 (77%)
|
Abbreviation: IOL, intraocular lens.
Resident preferences on the video-coaching curriculum were also evaluated and demonstrate that 100% of both observing (99 of 99) and presenting (13 of 13) residents had comfort presenting cases with none having a desire for an alternative setting. The majority of residents (10 of 13 presenting and 60 of 99 observing) felt the session was mentor driven.
Discussion
The video-coaching curriculum was developed and implemented over a 6-month-trial period. This study involved 6 senior residents presenting a total of 24 cases. All faculty and residents involved found benefit in the new curriculum, particularly in overall performance and avoiding errors. The curriculum allowed resident ownership of cases, self-reflection, teaching to other experience levels, and objective evaluation of individual surgeries.
In traditional ophthalmology residency programs, the majority of operations occur in the final 12 months of a resident's training. While there has been improvement in integrating surgery earlier into residency programs, this video-coaching curriculum makes surgical terminology, techniques, and discussions available to beginning residents and students. It helps maintain interest in one of the core aspects of ophthalmology and helps prepare less experienced residents for their operating time. In addition, it enhances the primary surgical year of senior resident physicians.
Residents spend up to 25% of their time teaching other residents and students and is a core component of residency training.[22] This curriculum provides another opportunity for resident physicians to develop techniques of reviewing and teaching to an audience. It provides an opportunity for residents to take ownership of cases in a safe and structured forum. Unlike traditional morbidity and mortality surgical conferences, these sessions provide the novice surgeon the opportunity to both teach and learn aspects of surgery that are only available in reflective practice. Faculty went to great lengths to keep the environment unintimidating, nonthreatening, and collegial to facilitate discussion of complications. The acquired data illustrate the comfort of residents with presenting in this setting and their desire for no changes to the current curriculum.
In addition to the educational benefit, this particular learning experience fulfills several of the ACGME's institutional expectations for patient safety, health care quality, and supervision as required by the CLER pathways to excellence. By focusing on techniques to improve surgical skills and improve recognition of difficult surgical situations, this curriculum addresses operating room system issues as well as approaches to prevent further problems. By providing direct feedback on individual surgeries in this setting, residents have the opportunity to receive additional input on their previous experiences to prepare for future surgeries.
While designing educational curricula, time and cost is an important factor. Both resident and faculty physicians have limited time and resources requiring educational sessions to be efficient. This forum allows the review of a wide array of topics, from instruments to technique to near misses, to be discussed with a moderate to large group in 1 hour. While it does require at least one faculty member to facilitate the session with the senior resident, it provides surgical exposure that typically would require one-on-one attention.
This study also provided insight into resident physicians' ability to rate their own performance as well as that of their coresidents. In previous reviews, residents were more accurate at rating global indices of a performance rather than specific tasks with a more accurate assessment by more senior residents.[23] The outcomes from this study illustrated that presenting residents rated their performances lower than faculty or other observing residents. Specifically, residents scored themselves lowest in the category “time and motion.” These results may suggest a more critical eye when evaluating personal work with a desire for perfection and/or a lack of confidence. These sessions can be used to help residents develop a sense of their progress and provide confidence that they are progressing as surgeons. Similar to other surgical fields' experience, it is likely that this curriculum can also lead to a faster learning curve and improved residents' surgical experience.
Additional benefits of this curriculum include the ability to probe a resident's surgical understanding without interfering with patient's confidence. Surgeries in ophthalmology, unlike many other surgical fields, often occur with the patient awake. It is not appropriate in many situations to probe the residents' knowledge base due to patient concerns, but this setting provides an opportunity for that learning modality.
This study and evaluation has several limitations. While it provides a framework for a video-coaching curriculum that can be utilized by other residency programs, it does not directly compare outcomes in a controlled fashion. Further research could include a randomized controlled study evaluating the outcomes after implementation of a similar curriculum. Additionally, the pilot sample size was small with only 24 surgical case presentations, though there were significantly more corresponding questionnaires completed. To widen the reach and utility of this program, a live feed for virtual observers or video recording with dissemination may be beneficial and considered in the future.
Conclusion
A formal video-coaching curriculum in ophthalmology provides a residency program with a unique opportunity to help trainees of all levels. In all participants, there was a consensus that the curriculum was beneficial for cataract surgery preparation. While there were differences in how presenting residents rated their performance, all participants were comfortable taking part in the curriculum and did not prefer an alternative curriculum.
