Keywords
resident education - cataract surgery - video coaching - curriculum - surgical curriculum
Medical and surgical training has long followed an apprenticeship model with the historical
teaching of “See one, do one, teach one.”[1]
[2] As medicine has advanced, both technologically and in the ability to measure patient
outcomes, there has been a push toward competency-based assessment. Starting in 1998,
the American Council on Graduate Medical Education (ACGME) developed six areas of
competency that residents had to meet: patient care, medical knowledge, practice-based
learning, interpersonal and communication skills, professionalism, and systems-based
practice. The ACGME has also recently instituted the CLER (Clinical Learning Environment
Review) Pathways to Excellence. Rather than simply counting cases, monitoring training
outcomes became required and expectations on topics such as patient safety, health
care quality, and supervision were increased.[3]
[4]
[5]
[6]
More than 3 million cataract surgeries are performed in the United States annually[7] and cataract surgery is the most commonly performed intraocular surgery by residents
during residency training. Currently, the minimum number of cataract surgeries required
by the ACGME is 86 with most programs easily exceeding this number.[8] After the ACGME mandate, there were many studies performed to evaluate resident
outcomes, methods of teaching cataract surgery, and the cost of resident educational
modalities.[3]
[9]
[10]
[11]
[12]
[13] There was a demonstrable learning curve for phacoemulsification cataract surgery
with reductions in complications and improvements in efficiency with more experience.[14] Newer methodologies of training and learning include wet laboratory curricula,[9]
[12] virtual reality training,[15]
[16] surgical simulators such as EyeSi, and tools to evaluate progression,[3]
[10]
[11] all with the goal of having surgeons master the learning curve earlier with fewer
complications. When discussing these new modalities, cost has been an increasingly
important consideration to ensure ability for implementation, sustainability, and
positive net benefit.[13]
One area of training that many other surgical fields, including general and orthopedic
surgery, have adapted is personalized video-based feedback to improve performance
in surgeries.[17]
[18] Reflective surgical practice has long been effective in enhancing education as it
has been used at conferences, presentations, and research forums.[19] Furthermore, video review often allows root cause analysis to learn from habits
or mistakes that lead to system errors.[20] In a recent general surgery study, comprehensive surgical coaching enhanced the
surgery training and led to superior skill acquisition when compared with conventional
training.[21]
Learning through simulators, operating room experience, and other methods may be enhanced
by a formal curriculum, including objective assessment of actual surgical cases, debriefing,
feedback from multiple levels of surgeons, and self-reflection. The aim of this study
was to develop, implement, and evaluate a video-coaching curriculum for cataract surgery
in a U.S. residency program.
Methods
A group of experienced faculty members met to develop a 6-month pilot video-coaching
curriculum for the Ophthalmology Residency Program at Emory University in Atlanta,
Georgia.
The developed curriculum included bi-weekly video conferences running from January
2016 through June 2016. At each session, one or two senior residents showed a surgical
video he/she performed with a faculty surgeon demonstrating a specific topic. The
cases were chosen by the resident in consultation with the faculty surgeon. Topics
were focused on either an intraoperative complication or a complex case requiring
advanced techniques. The sessions were 1 hour long and performed in an open format
with the video running at one to two times the real speed. The video was projected
on a 10 × 8 ft screen so that all participants could view the surgery footage. The
resident and faculty physician went through the case in a step-by-step fashion with
frequent pauses for discussion. Attendees included faculty physicians and individuals
in all levels of training such as resident physicians (PGY2–4) and medical students.
The faculty mentor would ensure that all important aspects of the case were reviewed.
In addition to the resident physician describing his/her thought process on certain
steps, the faculty physician provided tips and pitfalls to avoid. Frequent questions
were asked of the residents and students in the audience with input from all faculty
physicians. The environment was designed to be collaborative, nonjudgmental, and educational.
Each operating resident, faculty attending surgeon, and observing resident filled
out a post-session questionnaire ([supplementary Figs. S1] and [S2]). The questionnaires included questions on video coaching, trainee's performance,
mentor's style, reasons for stalls, and an Objective Structured Assessment of Technical
Skill (OSATS) evaluation of the surgery. Likert scales were used to provide quantitative
data as answers for the majority of the questions with a few questions permitting
qualitative feedback. The questionnaire responses were not made available to the presenting
resident, the audience, or the attending physicians and the results had no bearing
on judging the resident's surgical proficiency. This was made clear to all in attendance
at the beginning of each session.
The questionnaire response data were entered into an Excel spreadsheet and statistical
analyses were performed between observing resident, presenting resident, and faculty
attendee questionnaires. Analyses included mean and standard deviation comparisons,
correlation coefficients, and Student's t-tests.
Results
Through the 6-month pilot study, 6 senior resident physicians presented a total of
24 cases during 12 sessions. There were 13 presenting resident questionnaire responses,
99 observing resident responses, 2 medical student responses, and 35 faculty responses
after the 12 sessions. There were four faculty members with at least one of the faculty
present at each session to provide faculty responses.
[Table 1] illustrates the OSATS scores, including scores for each individual category and
the overall average score for each participating group. Presenting residents have
a lower average score of 3.32 when compared with both observing residents' score of
4.14 (p < 0.001) and faculty physicians' score of 4.20 (p < 0.001). There is no significant difference between observing resident and faculty
scores.
Table 1
Post-session objective structured assessment of technical skill (OSATS) scores
|
Respect for tissue
|
Time and motion
|
Knowledge and handling of instrument
|
Flow of operation
|
Use of assistants
|
Knowledge of specific procedure
|
Average score
|
|
Presenting residents
|
3.31
|
3.00
|
3.54
|
3.23
|
3.54
|
3.31
|
3.32
|
|
Observing residents
|
4.09
|
4.04
|
4.26
|
4.08
|
4.18
|
4.18
|
4.14
|
|
Faculty
|
4.19
|
4.09
|
4.26
|
4.13
|
4.28
|
4.27
|
4.20
|
Notes: p-Values are <0.001 for presenting residents versus observing residents and presenting
residents versus faculty. p-Value is 0.18 and is not statistically significant for observing residents versus
faculty.
The perceived benefits of the video-coaching curriculum are demonstrated in [Table 2]: 100% of residents and faculty found the curriculum beneficial. The majority of
participants in each category found benefit in recall of technical cues (11 of 13
presenting residents, 76 of 99 observing residents, 29 of 35 faculty), avoiding errors
(12 of 13 presenting residents, 81 of 99 observing residents, 31 of 35 faculty), flow
of performance (9 of 13 presenting residents, 50 of 99 observing residents, 27 of
35 faculty), recall of procedural steps (8 of 13 presenting residents, 56 of 99 observing
residents, 26 of 35 faculty), and overall performance (12 of 13 presenting residents,
66 of 99 observing residents, 31 of 35 faculty). The majority of presenting residents
and faculty found benefit in time of task (7 of 13 and 28 of 35, respectively), but
observing residents (36 of 99) did not perceive this as a benefit.
Table 2
Benefits of video coaching
|
Presenting residents (13)
|
Observing residents (99)
|
Faculty (35)
|
|
Overall benefit
|
13 (100%)
|
99 (100%)
|
35 (100%)
|
|
Recall of technical cues
|
11 (85%)
|
76 (77%)
|
29 (83%)
|
|
Time of task
|
7 (53%)
|
36 (36%)
|
28 (80%)
|
|
Avoiding errors
|
12 (92%)
|
81 (82%)
|
31 (89%)
|
|
Flow of performance
|
9 (69%)
|
50 (51%)
|
27 (77%)
|
|
Recall of procedural steps
|
8 (62%)
|
56 (57%)
|
26 (74%)
|
|
Overall performance
|
12 (100%)
|
66 (67%)
|
31 (89%)
|
[Table 3] illustrates the participants' overall performance rating and the resulting statistical
analysis. Observing residents and faculty had similar average score (1.7 and 1.6,
respectively) with the majority choosing excellent (54 of 99 and 20 of 35, respectively),
while presenting residents overwhelmingly chose satisfactory (8 of 13) with a statistically
significant (p < 0.001) lower average score (2.5).
Table 3
Post-session overall performance
|
Presenting residents (13)
|
Observing residents (99)
|
Faculty (35)
|
|
Excellent (1)
|
2 (15%)
|
54 (55%)
|
20 (57%)
|
|
Average (2)
|
2 (15%)
|
22 (22%)
|
12 (34%)
|
|
Satisfactory (3)
|
8 (62%)
|
22 (22%)
|
0 (0%)
|
|
Unsatisfactory (4)
|
1 (8%)
|
1 (1%)
|
3 (8.6%)
|
|
Average 4-point score
|
2.5
|
1.7
|
1.6
|
Notes: p-Values are <0.001 for presenting residents versus observing residents and presenting
residents versus faculty. p-Value is 0.28 and is not statistically significant for observing residents versus
faculty.
[Table 4] illustrates the comfort level of the various groups with residents performing different
steps of cataract surgery: 100% of presenting residents (13 of 13) felt comfortable
with each individual step of the surgical procedure, except nucleus disassembly where
85% (11 of 13) noted comfort. Attending physicians had varying comfort levels with
86% (30 of 35) expressing confidence for nucleus disassembly, quadrant removal, and
cortical clean-up, while 77% (27 of 35) had comfort with closure, 89% (31 of 35) had
comfort with intraocular lens insertion, 91% (32 of 35) had comfort with capsulorrhexis,
and 100% (35 of 35) had comfort with incisions.
Table 4
Comfort with resident performing task autonomously
|
Presenting residents (13)
|
Observing residents (99)
|
Faculty (35)
|
|
Incisions
|
13 (100%)
|
75 (76%)
|
35 (100%)
|
|
Capsulorrhexis
|
13 (100%)
|
53 (54%)
|
32 (91%)
|
|
Nucleus disassembly
|
11 (85%)
|
49 (49%)
|
30 (86%)
|
|
Quadrant removal
|
13 (100%)
|
45 (45%)
|
30 (86%)
|
|
Cortical clean up
|
13 (100%)
|
46 (46%)
|
30 (86%)
|
|
IOL insertion
|
13 (100%)
|
70 (71%)
|
31 (89%)
|
|
Closure
|
13 (100%)
|
65 (66%)
|
27 (77%)
|
Abbreviation: IOL, intraocular lens.
Resident preferences on the video-coaching curriculum were also evaluated and demonstrate
that 100% of both observing (99 of 99) and presenting (13 of 13) residents had comfort
presenting cases with none having a desire for an alternative setting. The majority
of residents (10 of 13 presenting and 60 of 99 observing) felt the session was mentor
driven.
Discussion
The video-coaching curriculum was developed and implemented over a 6-month-trial period.
This study involved 6 senior residents presenting a total of 24 cases. All faculty
and residents involved found benefit in the new curriculum, particularly in overall
performance and avoiding errors. The curriculum allowed resident ownership of cases,
self-reflection, teaching to other experience levels, and objective evaluation of
individual surgeries.
In traditional ophthalmology residency programs, the majority of operations occur
in the final 12 months of a resident's training. While there has been improvement
in integrating surgery earlier into residency programs, this video-coaching curriculum
makes surgical terminology, techniques, and discussions available to beginning residents
and students. It helps maintain interest in one of the core aspects of ophthalmology
and helps prepare less experienced residents for their operating time. In addition,
it enhances the primary surgical year of senior resident physicians.
Residents spend up to 25% of their time teaching other residents and students and
is a core component of residency training.[22] This curriculum provides another opportunity for resident physicians to develop
techniques of reviewing and teaching to an audience. It provides an opportunity for
residents to take ownership of cases in a safe and structured forum. Unlike traditional
morbidity and mortality surgical conferences, these sessions provide the novice surgeon
the opportunity to both teach and learn aspects of surgery that are only available
in reflective practice. Faculty went to great lengths to keep the environment unintimidating,
nonthreatening, and collegial to facilitate discussion of complications. The acquired
data illustrate the comfort of residents with presenting in this setting and their
desire for no changes to the current curriculum.
In addition to the educational benefit, this particular learning experience fulfills
several of the ACGME's institutional expectations for patient safety, health care
quality, and supervision as required by the CLER pathways to excellence. By focusing
on techniques to improve surgical skills and improve recognition of difficult surgical
situations, this curriculum addresses operating room system issues as well as approaches
to prevent further problems. By providing direct feedback on individual surgeries
in this setting, residents have the opportunity to receive additional input on their
previous experiences to prepare for future surgeries.
While designing educational curricula, time and cost is an important factor. Both
resident and faculty physicians have limited time and resources requiring educational
sessions to be efficient. This forum allows the review of a wide array of topics,
from instruments to technique to near misses, to be discussed with a moderate to large
group in 1 hour. While it does require at least one faculty member to facilitate the
session with the senior resident, it provides surgical exposure that typically would
require one-on-one attention.
This study also provided insight into resident physicians' ability to rate their own
performance as well as that of their coresidents. In previous reviews, residents were
more accurate at rating global indices of a performance rather than specific tasks
with a more accurate assessment by more senior residents.[23] The outcomes from this study illustrated that presenting residents rated their performances
lower than faculty or other observing residents. Specifically, residents scored themselves
lowest in the category “time and motion.” These results may suggest a more critical
eye when evaluating personal work with a desire for perfection and/or a lack of confidence.
These sessions can be used to help residents develop a sense of their progress and
provide confidence that they are progressing as surgeons. Similar to other surgical
fields' experience, it is likely that this curriculum can also lead to a faster learning
curve and improved residents' surgical experience.
Additional benefits of this curriculum include the ability to probe a resident's surgical
understanding without interfering with patient's confidence. Surgeries in ophthalmology,
unlike many other surgical fields, often occur with the patient awake. It is not appropriate
in many situations to probe the residents' knowledge base due to patient concerns,
but this setting provides an opportunity for that learning modality.
This study and evaluation has several limitations. While it provides a framework for
a video-coaching curriculum that can be utilized by other residency programs, it does
not directly compare outcomes in a controlled fashion. Further research could include
a randomized controlled study evaluating the outcomes after implementation of a similar
curriculum. Additionally, the pilot sample size was small with only 24 surgical case
presentations, though there were significantly more corresponding questionnaires completed.
To widen the reach and utility of this program, a live feed for virtual observers
or video recording with dissemination may be beneficial and considered in the future.
Conclusion
A formal video-coaching curriculum in ophthalmology provides a residency program with
a unique opportunity to help trainees of all levels. In all participants, there was
a consensus that the curriculum was beneficial for cataract surgery preparation. While
there were differences in how presenting residents rated their performance, all participants
were comfortable taking part in the curriculum and did not prefer an alternative curriculum.
