Introduction
The education and training of gastroenterology fellows, particularly during endoscopy,
can be challenging for both the trainee and preceptor. Prior studies that define acquisition
of endoscopic competency and evaluation of various instructional methods such as simulator
education are well-established [1]
[2]
[3]
[4]. Despite these publications, no study focuses directly on evaluation of the endoscopic
mentoring relationship. The establishment of an effective teaching/mentoring relationship
can significantly impact overall endoscopic training and subsequent performance. Effective
mentoring ideally takes place in an endoscopy suite with full attention of the preceptor
and minimal distraction. A critical part of mentoring involves uninterrupted communication
and focus; however, multiple sources of distraction (computers, cell phones, tablets,
pagers etc.) are abundant, and can negatively impact the endoscopic mentoring experience.
Moreover, prior studies have shown that communication failures within healthcare institutions
can increase the likelihood of adverse events [5]. Distractions in procedural areas can create misunderstandings and result in otherwise
preventable errors [6].
There has been evidence suggesting that a “sterile cockpit approach” used in aviation
can help minimize distractions. The Sterile Cockpit Rule is a Federal Aviation Administration
(FAA) regulation requiring aircraft pilots to refrain from non-essential activities
in the cockpit during critical phases of flight [7]. Prior studies have noted that implementation of the sterile cockpit methodology
led to a significant reduction in observed interruptions and improved awareness of
distractions in the gastroenterology procedural area. Furthermore, a recent study
identified distractedness as a target to address given the potential relationship
with adenoma detection rate (ADR) [8]
[9].
While no prior studies have directly evaluated the impact of distraction on endoscopic
training in a gastroenterology unit, some have shown detrimental effects on teamwork,
workload, and stress on members of operating room teams [10]. Conversely, one prior study used a novel tablet application to focus mentor attention
on trainee polyp detection, which resulted in reduced stress, quicker polyp identification,
and improved educational satisfaction [11].
To our knowledge, the use of distraction elimination/minimization has not been studied
in the setting of endoscopic training. Our study was designed to evaluate the impact
of distraction elimination/minimization on endoscopic mentoring and performance in
a clinical setting. We hypothesized that distraction minimization would improve perceived
endoscopic mentoring for fellows and attendings and improve endoscopic performance
metrics.
Methods
Overview
A prospective, randomized, crossover study was conducted of fellow and attending physicians
performing esophagogastroduodenoscopy (EGD) and colonoscopy (CLS) at the Tennessee
Valley Healthcare System (TVHS) Nashville Veterans Affairs Hospital in Nashville,
TN from January 2019 through May 2019. The protocol was IRB approved (TVHS IRB [protocol
number 1298089-1 under Title 38 CFR 16.101(b) Category 2]).
Participants, interventions and room design
Eight fellow physicians and seven attending physicians were enrolled to participate
in the study. Each was assigned a unique letter identification code to ensure anonymity
in the data acquisition process. Basic fellow/attending demographics without personal
identifying information was recorded. Participation in the study was voluntary and
all participants retained the right to withdraw from the study at any time. The study
was IRB approved for implied consent and a descriptive summary of the study was presented
prior to study commencement, but otherwise study personnel did not interact with participants.
No patient information was collected for the study. The authors of the paper were
not permitted to participate in endoscopy. Exclusion criteria for cases included surgically
altered anatomy (i.e prior history of ileo-cecectomy or hemicolectomy).
For each procedure, one fellow worked with one attending who was present for the entire
duration of the endoscopy. Each fellow was assigned to different attendings throughout
the study period. Prior to procedure initiation, each fellow/attending pair was randomized
to either “distraction minimization” (DM) or “standard room” (S) via a sealed envelope
prior to room entry ([Fig. 1]). Randomization sequence was generated via random number generator. In the “distraction
minimization” rooms, fellow/attending use of any electronic device was prohibited.
Secured, opaque, vibration-proof boxes were provided in each room to place phones,
pagers, tablets, or smart watches. Physician computer monitors were turned off during
the entire duration of the procedure. Non-essential staff were not permitted in the
procedure rooms, and no personnel were permitted to enter or leave the endoscopy room
once a procedure started unless necessitated by the case (i. e. additional equipment
or personnel needed). A sign was posted outside of each procedure room alerting the
staff of entry restrictions. No music was permitted in the rooms. In “standard rooms,”
there were no restrictions and the procedures took place under typical daily conditions.
Fig. 1 Daily room assignment protocol.
After completion of each procedure, the fellow and attending independently completed
separate forms and returned them to a secure location. The forms were collected by
research staff who were not directly involved with the procedure or in direct contact
with the study participants. Data was subsequently entered into a secure data repository
(Vanderbilt University Medical Center Redcap database (REDCap; UL1 TR000445 from NCATS/NIH)
[12].
Objectives/outcomes
The main outcome measure was fellow “Satisfaction with Training and Mentoring” as
measured by a previously validated 100 point visual analogue scale (VAS) [13]. Additional outcomes measured included “Ease of Communication,” “Ease of Identifying
pathology/Anatomic Landmarks,” “Satisfaction with Attentiveness of Attending Physician,”
and “Level of Distractedness.” Similarly, attending physicians completed a similar
post procedure survey instrument that measured “Overall Satisfaction with Mentoring
Experience,” “Ease of Communication,” “Level of Distractedness,” and “Level of Attentiveness.”
Endoscopic performance metrics were collected for each case and included successful
examination of the esophagus, stomach, and duodenum (including retroflexion in the
stomach) for upper endoscopies and cecal intubation rate, cecal intubation time, withdrawal
time, total procedure time, need for hands on attending assistance, and polyp detection
rate (PDR) for colonoscopies. Cecal intubation time was defined as the difference
in time between time of anal insertion of colonoscope and first visualization of the
appendiceal orifice (AO). Withdrawal time was defined as the difference between time
of visualization of AO and procedure end time. Total procedure time was defined as
the difference between time of colonoscope insertion and procedure end time. Polyp
detection rate was defined as the proportion of colonoscopies in which at least one
polyp is detected for each endoscopist.
Statistical analysis
We planned a study with 20 pairs of subjects. We estimated that the difference in
the response of matched pairs is normally distributed with a standard deviation of
12 (this assumes a standard deviation of 20 with a correlation among repeated measurements
on the same fellow of 0.7). Using this framework, we are able to detect a true difference
in the mean response of matched pairs of –9.179 or 9.179 with probability (power)
0.9 and detect a true difference in the mean response of matched pairs of –7.926 or
7.926 with probability (power) 0.8. The Type I error probability associated with this
test of the null hypothesis is 0.05. Each outcome considered was analyzed and processed
in a similar manner. The outcome was averaged over the type of room (DM or S) for
each fellow and attending to create an average score in both types of room for each
person. Paired t-tests were used to compare a subject’s average score in the DM room
to the same subject’s average score in the S room. Results are presented as mean differences
over all subjects (DM minus S) with corresponding 95 % CI to determine if the type
of room was associated with a change in outcome allowing for each subject to serve
as his/her own control with equal weight in the analysis.
Results
Fellow and attending characteristics
A total of eight fellow physicians (5 males, 3 females) and seven attending physicians
(5 males, 2 females) participated in the study. Specific details regarding each endoscopists
demographics and prior experience is included in [Table 1].
Table 1
Endoscopist demographics.
|
Fellow
|
Gender
|
Year in fellowship training
|
Total prior EGDs
|
Total prior colonoscopies
|
|
G
|
Female
|
1
|
50–99
|
21–49
|
|
Q
|
Female
|
1
|
26–49
|
21–49
|
|
O
|
Female
|
1
|
50–99
|
50–99
|
|
F
|
Male
|
2
|
> 200
|
> 200
|
|
P
|
Male
|
2
|
> 200
|
> 200
|
|
T
|
Male
|
2
|
> 200
|
> 200
|
|
S
|
Male
|
2
|
> 200
|
> 200
|
|
U
|
Male
|
3
|
> 200
|
> 200
|
|
Attending
|
Gender
|
Total years in practice
|
Total years mentoring fellows
|
Total EGDs
|
Total colonoscopies
|
|
A
|
Male
|
20
|
13
|
> 1000
|
> 1000
|
|
B
|
Male
|
20
|
20
|
> 1000
|
> 1000
|
|
C
|
Female
|
3.5
|
3.5
|
> 1000
|
> 1000
|
|
E
|
Female
|
11
|
11
|
> 1000
|
> 1000
|
|
H
|
Male
|
9
|
9
|
> 1000
|
> 1000
|
|
J
|
Male
|
30
|
27
|
> 1000
|
> 1000
|
|
K
|
Male
|
10
|
10
|
> 1000
|
> 1000
|
EGD, esophagogastroduodenoscopy.
Satisfaction with mentoring experience and endoscopic performance metrics
A total of 164 procedures were performed by the fellow/attending pairs (EGDs = 66,
Colonoscopies = 98). Despite a trend toward less overall distraction between rooms,
there was no statistically significant difference in overall fellow satisfaction with
training/mentoring, attending attentiveness, identifying pathology/anatomic landmarks
or ease of communication ([Table 2]). Notably, for the “level of distractedness” VAS, a score of “0” meant least distracted,
and a score of “100” meant most distracted. This reversal of scale was clearly noted
in the post procedural survey. Similarly, there was no significant difference for
any outcome measure for attendings ([Table 2]).
Table 2
Satisfaction of fellows and attendings with mentoring experience.
|
Group
|
Visual analogue scale
|
Distraction minimization room (0 = min, 100 = max)[1]
|
Standard room (0 = min, 100 = max)[1]
|
Mean difference (MD) between rooms
|
P value
|
95 %confidence interval
|
|
Fellows (n = 8)
|
Overall satisfaction with training/ mentoring
|
93
|
93
|
-0.04
|
0.97
|
–2.46–2.37
|
|
Ease of Communication
|
95
|
95
|
1.00
|
0.37
|
–1.5–3.5
|
|
Ease of Identifying pathology/anatomic landmarks
|
94
|
94
|
–1.73
|
0.56
|
–8.5–5.0
|
|
Level of distractedness[2]
|
12.5
|
18.3
|
-4.12
|
0.17
|
–10.5–2.2
|
|
Satisfaction with attentiveness of attending physicians
|
95
|
92
|
0.86
|
0.77
|
–5.8–7.5
|
|
Attendings (n = 7)
|
Overall satisfaction with mentoring experience
|
94
|
91
|
1.38
|
0.61
|
–-4.9–7.7
|
|
Ease of communication
|
92
|
94
|
1.56
|
0.56
|
–4.6–7.7
|
|
Level of distractedness
|
14.6
|
20.0
|
–1.56
|
0.59
|
–8.3–5.22
|
|
Level of attentiveness
|
92
|
88
|
0.47
|
0.61
|
–1.6–2.5
|
1 Scoring scale is from 0 to 100 where “0” is the minimum and “100” is the maximum.
Fellow scores are the mean for all fellows; attending scores are the mean for all
attendings.
2 For level of distractedness, a score of “0” meant least distracted, and a score of
“100” meant most distracted
Performance metrics between the two groups were also similar and no statistically
significant difference was noted between rooms for cecal intubation time (DM = 6 min
42 s, S = 8 min 42 s, MD =–1.58, p = 0.44, [95 % CI, –6.1–3.0]), withdrawal time (DM = 19 min,
S = 20 min, MD = 1.25, p = 0.63, [95 % CI, –4.6–7.1]), total procedure time (DM = 27 min,
S = 29 min, MD = –0.23, p = 0.92, [95 % CI, –5.3–4.9]), or PDR (DM = 73 %, S = 74 %,
MD = 0.02, P = 0.85, [95 % CI,–0.25–0.30]) ([Table 3]). There was no difference in successful completion of EGD or cecal intubation as
both arms in each category were 100 % among all fellows. In addition, there was no
statistically significant difference regarding need for attending assistance ([Table 4]).
Table 3
Performance metrics.
|
Metric
|
Distraction minimization room (n = 8)[1]
|
Standard room (n = 8)[1]
|
Mean difference (MD)
|
P value
|
|
EGD completion rate (%)
|
100
|
100
|
0.00
|
1.0
|
|
Cecal intubation rate (%)
|
100
|
100
|
0.00
|
1.0
|
|
Withdrawal time (minutes)
|
19
|
20
|
1.25
|
0.63
|
|
Cecal intubation time (minutes:seconds)
|
6:42
|
8:42
|
–1.58
|
0.44
|
|
Total procedure time (minutes)
|
27
|
29
|
–0.23
|
0.92
|
|
Polyp detection rate (%)
|
73
|
74
|
0.02
|
0.85
|
EGD, esophagogastroduodenoscopy.
1 All scores are averaged over fellows.
Table 4
Need for attending assistance.
|
Percentage of the total endoscopic time that required attending assistance (%)
|
DM cases requiring attending assistance (n; %)
|
S cases requiring attending assistance (n; %)
|
|
0
|
7 (87.5)
|
6 (75)
|
|
5
|
1 (12.5)
|
0
|
|
10
|
0
|
1 (12.5)
|
|
100
|
0
|
1 (12.5)
|
DM, distraction minimization; S, standard.
Discussion
Effective mentoring of trainees in endoscopy requires active attention by the preceptor
in an educational environment. As multiple sources of frequent distraction are common
in endoscopy suites during training, we set to evaluate the impact of distraction
minimization on endoscopic mentoring and performance. Studies regarding the impact
of distraction minimization on training have been nearly vacant from the literature.
Most studies involving distraction minimization reflect on patient safety outcomes,
which have shown promise; however, none of the studies show that distraction minimization
had impact on proficiency metrics (i.e complication rates, length of hospital stay,
length of procedure). Our study was unique in that it: (1) Used a prospective cross
over design to compare fellows to themselves in two different mentoring environments;
(2) studied the effect of distraction minimization on trainee satisfaction with mentoring;
and (3) studied the impact of distraction minimization on well-defined endoscopic
proficiency metrics.
Contrary to our hypothesis, we found that distraction minimization did not improve
fellow or attending mentoring outcomes despite trends towards less distraction. Notably,
however, while there was no difference in mentoring or performance outcomes between
DM and S rooms, both arms had high overall mentoring scores (all scores were > 90)
and performance outcomes (relatively quick cecal intubation times, low total procedure
times, high cecal intubation rate, high EGD completion rate, and high PDR).
When examining individual trends for fellow distraction, six fellows noted reduced
levels of distraction, while two (both second-year fellows) reported increased distraction
while in DM rooms compared to S rooms ([Fig. 2]). Given the presumed increased experience/competence relative to more junior fellows
in the study, it is possible that increased mentor attention itself may have been
more distracting. The results may suggest that more experienced fellows prefer a more
autonomous environment when performing endoscopy and inherently become more distracted
when another endoscopist is present. In addition, as attendings were not preoccupied
by phones or computers, there may have been more conversation between fellows and
preceptors causing more fellow distraction during endoscopy.
Fig. 2 Individual distraction trends. a Change in fellow reported distraction. Each line represents a fellow. b Change in attending reported distraction. Each line represents an attending.
We found similar results for attendings. While pooled results showed less overall
distraction, three of the seven attendings trended toward more distractedness in DM rooms
compared to S rooms – potentially as a result of them continuously thinking (consciously
or unconsciously) about what was happening outside of the room (i. e. getting behind
on emails; pages; EHR communications; etc.). Further studies to examine the impact
of these distractions are underway.
While the sample size was limited due to the number of fellows available for the study,
the results indicate that contrary to our hypothesis, distraction minimization may
have the opposite effect of that intended in some individuals – i. e. causing more
distraction by removing distractions. A matched study would be ideal; however, due
to logistics (several scheduling conflicts of fellows and attendings, vacation time,
medical leave, endoscopy lab closures etc.) this was prohibitive and therefore a cross-over
design was implemented a priori. While there are no prior or baseline studies to serve
as a reference for magnitude of improvement in distraction or satisfaction with training,
we felt that the crossover nature of the study served as an adequate comparison of
each fellow at baseline in an inherently more distracted environment.
Our study took place in the second half of the academic year, so junior fellows were
presumably more competent/experienced in endoscopy compared to a trial that started
at the beginning of the academic year. Perhaps a study evaluating the same variables
for a given trainee level stratified by time of academic year (i. e., first quarter
vs fourth quarter) may lead to changes in significance. Notably, however, a subgroup
analysis stratified by fellow training year showed no difference in any outcomes measured.
Also, given the inherent differences and learning curves for EGD and colonoscopy –
with colonoscopy generally considered to be more difficult to learn, we separately
evaluated the outcomes for each fellow for EGD and colonoscopy, and no differences
were noted.
We considered a mixed-method approach to include a post study survey; however, there
was overall concern for compliance with an additional survey required to obtain this
information. We did conduct informal debriefings with fellows and attending physicians
after the study was concluded. Overall, fellows felt that there were many times where
they felt paradoxically more distracted when the attendings were paying more attention
and giving more advice (welcomed or unwelcomed) during cases. While attendings were
not enthusiastic about surrendering access to their technology, they did feel more
engaged overall in the teaching process.
While the use of several mentors in this study may function as a confounder given
the subjective nature of outcomes, we felt that attending input about the educational
experience was very important and therefore included a variety of mentors in the analysis
to eliminate the chance that the results would be specific only to one mentor as different
attendings provide different levels of feedback/engagement/teaching.
While our study design successfully eliminated distractions pertaining to the attending
physicians and fellows, other distractions including technologists, nurses, and anesthesia
staff still existed. As adequate training and mentoring is mostly focused on the interactions
and distractedness of the fellow and attendings, we felt that distraction of others
in the room would have minimal influence on the results. Finally, given the nature
of the distraction minimization rooms and direct interaction with pairs of our study,
our study was not able to be blinded. Therefore, it is possible that attendings were
unconsciously/consciously more engaged in mentoring during procedures regardless of
which arm they were assigned (DM or S) – i. e. the Hawthorne effect.
Given the similarity in both mentoring and performance outcomes between DM and S rooms,
the results of this study suggest that simply having an attending provider physically
present during the entire duration of the procedure affects perceived mentoring and
performance – as attending providers were physically present and readily available
to help with questions and assist with procedures if needed in both study arms. While
some fellowship programs require attendings to be present in the endoscopy suite for
the entire duration of the procedure, others require that an attending physician be
“immediately available” to assist fellows during endoscopy. As distraction minimization
may improve engagement/availability, a multicenter trial comparing the two arms in
this study with a third arm of “attending availability” would be helpful to delineate
if further differences exist.
Conclusion
In summary, distraction minimization does not improve perceived quality of endoscopic
mentoring for fellows or attendings or fellow performance when compared to standard
instruction; but may improve attending engagement/availability. Our study suggests
that simply having an attending provider in the endoscopy room may enhance the endoscopic
educational experience when compared to having attendings immediately available. Studies
evaluating distraction minimization and attending engagement/availability are underway.
