Keywords prenatal detection rate - simulation-based medical education - fetal ultrasound -
ultrasound simulation - second-trimester screening
Introduction
For prenatal diagnostics, ultrasound (US) is the gold standard to assess the development
of the fetus, amniotic fluid, and feto-maternal blood flow [1 ]. Technical improvements via high-resolution US imaging and routine screening allow
diagnosis of minor and complex malformations [2 ]. However, the quality of care leaves room for improvement [3 ]
[4 ]. Despite advanced technology and regular scans, prenatal detection rates of fetal
malformations still remain low [4 ]
[5 ]. Health and development surveillance of pregnant women helps to decrease perinatal
morbidity and mortality [6 ]. An essential part of quality assurance is structured training for US novices, as
it is demanded by specialist societies [7 ]
[8 ]
[3 ]
[9 ]. Obstetrical US examinations are conducted by obstetricians/gynecologists (Ob/Gyn)
with teaching embedded in the clinical routine. The German Institute for Quality and
Efficiency in Health Care (IQWiG) analyzed prenatal US screening in terms of the successful
detection of fetal anomalies and concluded in 2008 that there is a positive association
between higher qualification of the examiner and the successful detection of fetal
anomalies [10 ]. In Germany, independent US practice is regulated by successful completion of a
specialist exam and the mandatory acquisition of a certain number of scans. For prenatal
US certification, 300 utero-placental-fetal scans are required [11 ] without quality control. Neither medical students nor residents receive US training
as part of their routine medical education. Residents are often lost with respect
to how and when to receive US education and experience when facing an increasing workload
[12 ]
[3 ]. Ob/Gyn residents are confronted with a discrepancy between their perceived level
of confidence in US examination and expectations placed on them during their clinical
work [13 ]. Standardization of US curricula is demanded by educational committees, as well
as medical students and physicians in training [7 ]
[12 ]
[3 ]. An approach to meet the challenges of US education and quality assurance is to
train novices in simulated settings until a certain level of proficiency is achieved
to begin clinical training [14 ]. Simulation-based ultrasound training (SIM-UT) is an effective teaching tool for
Ob/Gyn US skills [8 ]. SIM-UT can increase trainees’ accuracy with respect to locating standard planes,
organ measurements, and examination speed [15 ]
[16 ]. Trainees who underwent SIM-UT have significantly improved their clinical performance
and patient-perceived quality of care [9 ]
[17 ]. Simulation-based tests are as effective as live model-based evaluations for obstetrical
US skill assessment [18 ]
[19 ]
[20 ]. Little is known about the effectiveness of SIM-UT for detailed 2nd trimester US examinations [15 ]
[8 ] as the most important US scan in the detection of fetal anomalies [1 ]
[21 ]. We aimed to analyze whether SIM-UT is applicable to train US novices in complex
2nd trimester protocols. In addition, we aimed to assess the training time and intensity
needed for US novices to gain a certain level of professionalism. Finally, we wanted
to analyze the learning success of novices after 6 weeks of SIM-UT in comparison to
physicians and US experts.
Materials and methods
Study design and participants
The fetal simulation study III (FESIM III) was a prospective trial to assess the learning
progress of ultrasound novices in 2nd trimester scans in comparison to two reference groups. 11 medical students from year
2 to 6 were included as novices. The inclusion criteria were a) enrollment as a student
b) completion of the subject of anatomy, c) less than 10 hours of experience in gynecological
or obstetrical US. Additionally, as reference groups, 10 Ob/Gyn physicians and 10
fetal US experts were included separately ([Fig. 1 ]). A parallel trial (FESIM II) analyzed learning curves during SIM-UT for fetal echocardiography.
All medical students attended an introductory 90-minute seminar, followed by individual
training sessions with the US Mentor. Two training sessions were supervised by student
tutors giving a 30-minute introduction to each participant. The following training
sessions were completed by the participants alone. Student tutors were available for
questions. The novices completed a one-hour session twice a week for a total of 12
hours of self-training per person during the training period. The training focused
on the correct acquisition of standard planes as an essential basis for any sonographic
assessment. Trainees were able to learn the standard planes in pathologic cases during
individual SIM-UT. All tests were carried out on a healthy, virtual fetus and the
progress was analyzed in standardized tests every two weeks. Participants were asked
to obtain and freeze all 23 correct standard views as quickly as possible with a moving
fetus. Tests were carried out under real life examination conditions with all aids
of the simulator being removed. The points in time when participants froze or unfroze
an image or decided to abort the search within a certain standard plane were recorded.
In examination mode, the US simulator automatically rated the obtained standard view
as correct or incorrect after the completion of the examination. During the trial
we noticed that the simulator sometimes rated correct planes as incorrect. Therefore,
a second level of examination had to be implemented. If a captured plane was rated
as incorrect by the simulator, a scientific expert panel consisting of three fetal
US experts was considered to rate the obtained views either as correct or incorrect
independently by retrospective video rating. The expert panel was comprised of members
of the quality securing committee (“KVWL-Qualitätssicherungskommission”) certified at least level II according to the “Deutsche Gesellschaft für Ultraschall
in der Medizin” (DEGUM – German society for ultrasound in medicine). Planes were reevaluated
using the grades used in the German school system, ranging from “1 – very good” to
“6 – insufficient”. The average grade of the three standard planes was calculated.
Standard planes with average grades “1–4” were retrospectively re-evaluated as correctly
obtained.
Fig. 1 Flowchart of ultrasound simulation training within trial and reference groups.
Ultrasound Simulator
Our study was conducted on the Simbionix US Mentor ultrasound simulator (Simbionix,
Beit Golan, Israel) consisting of a mannequin, various sham US probes, a touchscreen
with a built-in PC, footswitch, and the “electronic box” as the central computing
device connected to the transmitter. The latter, a hand-sized box attached to the
back of the mannequin, emits a magnetic field, which is detected by built-in sensors
of the sham US probes. This enables the simulator to locate the exact position of
the probe and the simulator renders a real-time image of a simulated fetus. This image
calculation relies on a virtual fetus, that was generated using MRI volume data. The
utilization of a virtual fetus enables the US Mentor to simulate fetal movements while
scanning to create a realistic examination environment. This feature is unique to
the US Mentor. The virtual fetus also facilitates the simulation of various artifacts
and enables the implementation of malformations. The 2nd trimester module consisted of six modules, comprised of anatomy practice, one module
with a healthy randomly moving fetus, and four randomly moving fetuses with different
malformations. 23 standard views were selected for the practice of a detailed 2nd trimester scan. The Ob/Gyn module is designed in compliance with ISUOG guidelines.
All planes of the sonomorphological scan of the German Maternity Guidelines IIb, the
standard screening for 2nd trimester US, were included [1 ] ([Table 1 ]). All standard planes according to DEGUM level II necessary for a detailed scan
were included in the trial except for special heart planes which were analyzed in
a separate study (FESIM II) focusing on fetal echocardiography [22 ]. [Fig. 2 ] demonstrates the US Mentor and different training modes. The reference groups received
a questionnaire to quantify previous US experience, information regarding their professional
career, such as specialist degree or level of DEGUM qualification.
Table 1 Standard planes and corresponding organs according to DEGUM guidelines.
Region
Plane
Region
Plane
Fetal head
Midline facial profile (MFP)
Abdomen
Abdomen with stomach (Abd)
Coronal face (CF)
Umbilical ord insertion (Umb)
Orbits
Kidneys coronal (Kidcor)
Brain
Transventricular
Kidneys axial (Kidax)
Transthalamic
Bladder transverse
Transcerebellar
Limbs
Humerus
Spine
Neck sagittal
Radius + ulna (RadUl)
Spine sagittal
Hand
Thorax
Four-chamber view (4CV)
Femur
Diaphragm/left lung (LeftLung)
Tibia + fibula (TibFib)
Diaphragm/right lung (RightLung)
Foot
Placenta
Placenta
Fig. 2
a–d US simulator equipment, (b ) screen in learning mode: thalamic plane, (c ) captured image with gold standard: spine, (d ) captured plane rated as incorrect/not identified by simulator: cerebellar plane.
Statistical Analysis
After completion of the examinations, the time intervals needed by the participants
to obtain the correct standard views were calculated using Microsoft Excel. The rate
of appropriate images (RAI) was defined as the percentage of correctly obtained standard
planes out of all standard planes. Total time to completion (TTC) was defined as the
timespan needed by participants to obtain all 23 standard planes. Statistical analysis
was performed using IBM SPSS Statistics 27. Paired non-parametric tests (Friedman
and Wilcoxon) were used to compare RAI and TTC of the trial group between their three
examinations after 2, 4, and 6 weeks. With non-parametric unpaired tests (Kruskal-Wallis
and Mann-Whitney U), we compared performance values between novices, physicians, and
DEGUM experts. A value of p<0.05 was considered significant.
Results
11 medical students from year 2 to 6 were compared using a standardized test with
the two reference groups (A) and (B). The experience levels of the reference groups
for Ob/Gyn US are indicated in [Table 2 ]. The trial group benefitted from significant learning progress during the entire
training period. In terms of accuracy, the group reached a medium score of 94.9±6.7%
correctly obtained images after two weeks of SIM-UT and improved it to 98.0±3.6% after
four weeks of training. They were able to reach the physicians’ RAI after 4 weeks,
i.e., 8 hours of self-training (98.0±3.6% vs. 96.5±4.5%). All members of the DEGUM
expert group continued to have a higher RAI of 100%, even though this difference was
not statistically significant (p overall=0.062). However, there was a significant
difference in RAI between the students' first test and the expert group (p<0.008).
The TTC needed by the trial group to obtain all 23 standard planes shortened significantly
over the training period ([Fig. 3 ]). The score of 1238±283 seconds in the first test after two weeks of SIM-UT decreased
to 620±189 seconds (average time per plane 27 seconds) after six weeks of training
(p overall <0.001). After six weeks of self-training, the trial group scored a significantly
faster TTC than the reference group of physicians (620±189 vs. 1036±389 seconds, p<0.001)
while the expert group remained significantly faster in its overall performance (443±243
seconds, p<0.001). Similar results show the TTC of obtaining the planes for basic
fetal biometry (transthalamic, abdominal, femur, and placental planes) ([Fig. 4 ]). Regarding the time to obtain single standard planes, significant differences between
the groups can be found in 8/23 planes. In 4 out of 8 planes, the trial group performed
significantly faster than the physician group, while significant differences in time
to obtain single standard planes showed no significant difference between novices
and DEGUM experts for 20/23 2nd trimester standard planes after six weeks of SIM-UT ([Fig. 5 ]). There was a significant difference for two planes of the basic fetal biometry
planes, as DEGUM experts performed significantly faster than the reference (A) and
trial groups. There was an overall of 268 obtained images that had to be reevaluated
by the expert panel. Subsequently, 32 of the 268 images were evaluated as insufficient.
Table 2 Ultrasound experience of reference groups.
Specialists
Residents
Median of Ob/Gyn US scans
Minimum number of scans performed
Maximum number of scans performed
Total number of scans performed
Reference group (A)
Physician group, n=10
2
8
750
130
40,000
48,130
Reference group (B)
Expert group, n=10
10
0
15, 000
8, 000
80, 000
264, 000
Fig. 3 Total time to completion of all 23 planes by study group.
Fig. 4 Total time to completion of all basic planes by study group.
Fig. 5 Boxplot showing the distribution of total time to completion of single planes between
trial group (students), reference group (A) (physicians), and reference group (B)
(experts).
Discussion
Advantages of SIM-UT include familiarization with US machines, acquisition of hand-eye
coordination, anatomy standard planes, and examinations prior to clinical training
on patients [23 ]. In our study we aimed to analyze these effects in learning curves for an extended
detailed 2nd trimester examination protocol, since many fetal malformations present in planes
beyond measuring fetal biometry [6 ]
[5 ]. This is the first study to analyze training progress of US novices on a high-end
simulator with a moving virtual fetus. Former studies on SIM-UT criticized the lack
of fetal movement of US simulators [15 ]. The median experience of the physician group in our trial is about the same as
the German standard for specialists in obstetrics, thus reflecting the skill level
of basic US screening in Germany [11 ]. Their average group performance may be considered as representative for Ob/Gyn
physicians in Germany since the group included physicians ranging from 1st year residency to consultant level. The fact that the trial group outperformed the
physician group was unexpected and one major significant finding of our study. One
reason might be insufficient supervision or a lack of structured US training associated
with decreased US performance [23 ]. Some planes included in the protocol of the test were beyond basic screening and
thus not necessarily applied in physicians’ daily practice. Considering only standard
planes, the trial group reached the physicians’ skill level in the acquisition of
basic standard planes but outperformed them significantly in the acquisition of the
extended screening protocol. Therefore, residents, and even Ob/Gyn specialists in
basic care will most likely benefit from SIM-UT in addition to their clinical training
regardless of the number of performed scans upfront in their clinical routine. The
strengths of the study include the highly qualified expert group consisting of only
certified DEGUM experts with total experience of 264,000 US scans. Data from the standardized
hands-on test, which was collected to analyze learning progress and to measure the
performance of the three groups, discriminated between the different US competence
levels of the groups. Subsequently, the expert group performed significantly better,
while the physician group outperformed the trial group in terms of accuracy until
the trial group gained experience of 8 hours of SIM-UT. That indicates the realism
of the simulator, the possibility of assessing different US competence levels with
a standardized hands-on simulator test, and the feasibility of learning 2nd trimester scans by structured SIM-UT even with no to minimum experience. Simulation-based
examinations for the assessment have already been shown to be a useful method [18 ]
[19 ]
[20 ]. If competence levels and skills can be assessed in a reliable way with US simulators,
simulation-based tests could be an answer to long demanded accreditation and quality
assurance for prenatal US [7 ]. Another strength of our study is the length of the intervention period, providing
structured SIM-UT on a regular basis over six weeks, and the extent of the practiced
examination protocol. Throughout the training period, the trial group recorded a significant
decrease in TTC between each test. At the end of the training period, there were only
three standard views that were accomplished significantly faster by the expert reference
group. This crucial finding demonstrates that even very difficult standard views of
extended protocols can be learned with SIM-UT. The positive feedback matches previous
data about SIM-UT. SIM-UT is accepted among trainees in postgraduate training and
is considered an essential part of their US education [24 ]. A limitation was the number of trial participants, which had to be kept limited,
to allow enough time for training and skill progress testing. In detail, one simulator
was available and shared with a parallel study with another group of 11 participants.
Overall, 22 trainees each trained 2 hours per week on one simulator. Remaining time
capacities were utilized to test skill progress and to evaluate the reference groups.
As another limitation, we noticed a high rate of clinically properly obtained images
that were not assigned to the matching standard plane. 268 views did not meet the
simulator’s criteria whereas after expert panel review only 32/268 were classified
as clinically insufficient. This discrepancy raises the question how the simulator
evaluates the obtained images and when and in which situation the standard planes
are rejected. In our experience, the tolerance should be adjusted for the acceptance
of planes. Overall, simulators provide a safe learning space, with examinations being
reproducible and cost-efficient [25 ]. In a standardized simulated setting trainees can focus on obtaining correct standard
views to be able to detect fetal anomalies [26 ]. Individual different fetal and maternal physical characteristics can challenge
sonographers and have not yet been implemented into the simulated examination conditions.
Regardless of the latter, there is consensus regarding the efficacy and feasibility
of SIM-UT in Ob/Gyn US [15 ]
[16 ]
[8 ]
[9 ]
[17 ]. The transferability of SIM-UT to real patients is part of successor trials. The
quality of US examinations is significantly limited by the experience of the US operator,
regardless of being a physician or technical sonographer [7 ]. Most importantly, while training US skills independent of mode of training (simulator
vs. real patients), knowledge of US machines, technical pitfalls, and the knowledge
behind image generation should be an inevitable part of all US teaching [27 ]. Residents should have the opportunity to train on US simulators, as errors on real
patients can effectively be limited through SIM-UT [28 ]. Structured SIM-UT is a highly effective tool to learn basic and extended prenatal
US. While enabling significant skill acquisition within short periods, it provides
a safe and learner-centered atmosphere and can be conducted independently during medical
curricula and residency training programs. In an age of virtual technology, physicians
should benefit from these realistic simulations of diagnostic and therapeutic procedures,
in addition to their daily clinical practice [28 ]. Overall, standardized hands-on tests on US simulators are a reliable tool to assess
operator skills and assure quality standards of US examinations.
