Keywords prenatal detection rate - simulation-based medical education - fetal ultrasound - ultrasound simulation - second-trimester screening
Introduction
For prenatal diagnostics, ultrasound (US) is the gold standard to assess the development of the fetus, amniotic fluid, and feto-maternal blood flow [1 ]. Technical improvements via high-resolution US imaging and routine screening allow diagnosis of minor and complex malformations [2 ]. However, the quality of care leaves room for improvement [3 ]
[4 ]. Despite advanced technology and regular scans, prenatal detection rates of fetal malformations still remain low [4 ]
[5 ]. Health and development surveillance of pregnant women helps to decrease perinatal morbidity and mortality [6 ]. An essential part of quality assurance is structured training for US novices, as it is demanded by specialist societies [7 ]
[8 ]
[3 ]
[9 ]. Obstetrical US examinations are conducted by obstetricians/gynecologists (Ob/Gyn) with teaching embedded in the clinical routine. The German Institute for Quality and Efficiency in Health Care (IQWiG) analyzed prenatal US screening in terms of the successful detection of fetal anomalies and concluded in 2008 that there is a positive association between higher qualification of the examiner and the successful detection of fetal anomalies [10 ]. In Germany, independent US practice is regulated by successful completion of a specialist exam and the mandatory acquisition of a certain number of scans. For prenatal US certification, 300 utero-placental-fetal scans are required [11 ] without quality control. Neither medical students nor residents receive US training as part of their routine medical education. Residents are often lost with respect to how and when to receive US education and experience when facing an increasing workload [12 ]
[3 ]. Ob/Gyn residents are confronted with a discrepancy between their perceived level of confidence in US examination and expectations placed on them during their clinical work [13 ]. Standardization of US curricula is demanded by educational committees, as well as medical students and physicians in training [7 ]
[12 ]
[3 ]. An approach to meet the challenges of US education and quality assurance is to train novices in simulated settings until a certain level of proficiency is achieved to begin clinical training [14 ]. Simulation-based ultrasound training (SIM-UT) is an effective teaching tool for Ob/Gyn US skills [8 ]. SIM-UT can increase trainees’ accuracy with respect to locating standard planes, organ measurements, and examination speed [15 ]
[16 ]. Trainees who underwent SIM-UT have significantly improved their clinical performance and patient-perceived quality of care [9 ]
[17 ]. Simulation-based tests are as effective as live model-based evaluations for obstetrical US skill assessment [18 ]
[19 ]
[20 ]. Little is known about the effectiveness of SIM-UT for detailed 2nd trimester US examinations [15 ]
[8 ] as the most important US scan in the detection of fetal anomalies [1 ]
[21 ]. We aimed to analyze whether SIM-UT is applicable to train US novices in complex 2nd trimester protocols. In addition, we aimed to assess the training time and intensity needed for US novices to gain a certain level of professionalism. Finally, we wanted to analyze the learning success of novices after 6 weeks of SIM-UT in comparison to physicians and US experts.
Materials and methods
Study design and participants
The fetal simulation study III (FESIM III) was a prospective trial to assess the learning progress of ultrasound novices in 2nd trimester scans in comparison to two reference groups. 11 medical students from year 2 to 6 were included as novices. The inclusion criteria were a) enrollment as a student b) completion of the subject of anatomy, c) less than 10 hours of experience in gynecological or obstetrical US. Additionally, as reference groups, 10 Ob/Gyn physicians and 10 fetal US experts were included separately ([Fig. 1 ]). A parallel trial (FESIM II) analyzed learning curves during SIM-UT for fetal echocardiography. All medical students attended an introductory 90-minute seminar, followed by individual training sessions with the US Mentor. Two training sessions were supervised by student tutors giving a 30-minute introduction to each participant. The following training sessions were completed by the participants alone. Student tutors were available for questions. The novices completed a one-hour session twice a week for a total of 12 hours of self-training per person during the training period. The training focused on the correct acquisition of standard planes as an essential basis for any sonographic assessment. Trainees were able to learn the standard planes in pathologic cases during individual SIM-UT. All tests were carried out on a healthy, virtual fetus and the progress was analyzed in standardized tests every two weeks. Participants were asked to obtain and freeze all 23 correct standard views as quickly as possible with a moving fetus. Tests were carried out under real life examination conditions with all aids of the simulator being removed. The points in time when participants froze or unfroze an image or decided to abort the search within a certain standard plane were recorded. In examination mode, the US simulator automatically rated the obtained standard view as correct or incorrect after the completion of the examination. During the trial we noticed that the simulator sometimes rated correct planes as incorrect. Therefore, a second level of examination had to be implemented. If a captured plane was rated as incorrect by the simulator, a scientific expert panel consisting of three fetal US experts was considered to rate the obtained views either as correct or incorrect independently by retrospective video rating. The expert panel was comprised of members of the quality securing committee (“KVWL-Qualitätssicherungskommission”) certified at least level II according to the “Deutsche Gesellschaft für Ultraschall in der Medizin” (DEGUM – German society for ultrasound in medicine). Planes were reevaluated using the grades used in the German school system, ranging from “1 – very good” to “6 – insufficient”. The average grade of the three standard planes was calculated. Standard planes with average grades “1–4” were retrospectively re-evaluated as correctly obtained.
Fig. 1 Flowchart of ultrasound simulation training within trial and reference groups.
Ultrasound Simulator
Our study was conducted on the Simbionix US Mentor ultrasound simulator (Simbionix, Beit Golan, Israel) consisting of a mannequin, various sham US probes, a touchscreen with a built-in PC, footswitch, and the “electronic box” as the central computing device connected to the transmitter. The latter, a hand-sized box attached to the back of the mannequin, emits a magnetic field, which is detected by built-in sensors of the sham US probes. This enables the simulator to locate the exact position of the probe and the simulator renders a real-time image of a simulated fetus. This image calculation relies on a virtual fetus, that was generated using MRI volume data. The utilization of a virtual fetus enables the US Mentor to simulate fetal movements while scanning to create a realistic examination environment. This feature is unique to the US Mentor. The virtual fetus also facilitates the simulation of various artifacts and enables the implementation of malformations. The 2nd trimester module consisted of six modules, comprised of anatomy practice, one module with a healthy randomly moving fetus, and four randomly moving fetuses with different malformations. 23 standard views were selected for the practice of a detailed 2nd trimester scan. The Ob/Gyn module is designed in compliance with ISUOG guidelines. All planes of the sonomorphological scan of the German Maternity Guidelines IIb, the standard screening for 2nd trimester US, were included [1 ] ([Table 1 ]). All standard planes according to DEGUM level II necessary for a detailed scan were included in the trial except for special heart planes which were analyzed in a separate study (FESIM II) focusing on fetal echocardiography [22 ]. [Fig. 2 ] demonstrates the US Mentor and different training modes. The reference groups received a questionnaire to quantify previous US experience, information regarding their professional career, such as specialist degree or level of DEGUM qualification.
Table 1 Standard planes and corresponding organs according to DEGUM guidelines.
Region
Plane
Region
Plane
Fetal head
Midline facial profile (MFP)
Abdomen
Abdomen with stomach (Abd)
Coronal face (CF)
Umbilical ord insertion (Umb)
Orbits
Kidneys coronal (Kidcor)
Brain
Transventricular
Kidneys axial (Kidax)
Transthalamic
Bladder transverse
Transcerebellar
Limbs
Humerus
Spine
Neck sagittal
Radius + ulna (RadUl)
Spine sagittal
Hand
Thorax
Four-chamber view (4CV)
Femur
Diaphragm/left lung (LeftLung)
Tibia + fibula (TibFib)
Diaphragm/right lung (RightLung)
Foot
Placenta
Placenta
Fig. 2
a–d US simulator equipment, (b ) screen in learning mode: thalamic plane, (c ) captured image with gold standard: spine, (d ) captured plane rated as incorrect/not identified by simulator: cerebellar plane.
Statistical Analysis
After completion of the examinations, the time intervals needed by the participants to obtain the correct standard views were calculated using Microsoft Excel. The rate of appropriate images (RAI) was defined as the percentage of correctly obtained standard planes out of all standard planes. Total time to completion (TTC) was defined as the timespan needed by participants to obtain all 23 standard planes. Statistical analysis was performed using IBM SPSS Statistics 27. Paired non-parametric tests (Friedman and Wilcoxon) were used to compare RAI and TTC of the trial group between their three examinations after 2, 4, and 6 weeks. With non-parametric unpaired tests (Kruskal-Wallis and Mann-Whitney U), we compared performance values between novices, physicians, and DEGUM experts. A value of p<0.05 was considered significant.
Results
11 medical students from year 2 to 6 were compared using a standardized test with the two reference groups (A) and (B). The experience levels of the reference groups for Ob/Gyn US are indicated in [Table 2 ]. The trial group benefitted from significant learning progress during the entire training period. In terms of accuracy, the group reached a medium score of 94.9±6.7% correctly obtained images after two weeks of SIM-UT and improved it to 98.0±3.6% after four weeks of training. They were able to reach the physicians’ RAI after 4 weeks, i.e., 8 hours of self-training (98.0±3.6% vs. 96.5±4.5%). All members of the DEGUM expert group continued to have a higher RAI of 100%, even though this difference was not statistically significant (p overall=0.062). However, there was a significant difference in RAI between the students' first test and the expert group (p<0.008). The TTC needed by the trial group to obtain all 23 standard planes shortened significantly over the training period ([Fig. 3 ]). The score of 1238±283 seconds in the first test after two weeks of SIM-UT decreased to 620±189 seconds (average time per plane 27 seconds) after six weeks of training (p overall <0.001). After six weeks of self-training, the trial group scored a significantly faster TTC than the reference group of physicians (620±189 vs. 1036±389 seconds, p<0.001) while the expert group remained significantly faster in its overall performance (443±243 seconds, p<0.001). Similar results show the TTC of obtaining the planes for basic fetal biometry (transthalamic, abdominal, femur, and placental planes) ([Fig. 4 ]). Regarding the time to obtain single standard planes, significant differences between the groups can be found in 8/23 planes. In 4 out of 8 planes, the trial group performed significantly faster than the physician group, while significant differences in time to obtain single standard planes showed no significant difference between novices and DEGUM experts for 20/23 2nd trimester standard planes after six weeks of SIM-UT ([Fig. 5 ]). There was a significant difference for two planes of the basic fetal biometry planes, as DEGUM experts performed significantly faster than the reference (A) and trial groups. There was an overall of 268 obtained images that had to be reevaluated by the expert panel. Subsequently, 32 of the 268 images were evaluated as insufficient.
Table 2 Ultrasound experience of reference groups.
Specialists
Residents
Median of Ob/Gyn US scans
Minimum number of scans performed
Maximum number of scans performed
Total number of scans performed
Reference group (A)
Physician group, n=10
2
8
750
130
40,000
48,130
Reference group (B)
Expert group, n=10
10
0
15, 000
8, 000
80, 000
264, 000
Fig. 3 Total time to completion of all 23 planes by study group.
Fig. 4 Total time to completion of all basic planes by study group.
Fig. 5 Boxplot showing the distribution of total time to completion of single planes between trial group (students), reference group (A) (physicians), and reference group (B) (experts).
Discussion
Advantages of SIM-UT include familiarization with US machines, acquisition of hand-eye coordination, anatomy standard planes, and examinations prior to clinical training on patients [23 ]. In our study we aimed to analyze these effects in learning curves for an extended detailed 2nd trimester examination protocol, since many fetal malformations present in planes beyond measuring fetal biometry [6 ]
[5 ]. This is the first study to analyze training progress of US novices on a high-end simulator with a moving virtual fetus. Former studies on SIM-UT criticized the lack of fetal movement of US simulators [15 ]. The median experience of the physician group in our trial is about the same as the German standard for specialists in obstetrics, thus reflecting the skill level of basic US screening in Germany [11 ]. Their average group performance may be considered as representative for Ob/Gyn physicians in Germany since the group included physicians ranging from 1st year residency to consultant level. The fact that the trial group outperformed the physician group was unexpected and one major significant finding of our study. One reason might be insufficient supervision or a lack of structured US training associated with decreased US performance [23 ]. Some planes included in the protocol of the test were beyond basic screening and thus not necessarily applied in physicians’ daily practice. Considering only standard planes, the trial group reached the physicians’ skill level in the acquisition of basic standard planes but outperformed them significantly in the acquisition of the extended screening protocol. Therefore, residents, and even Ob/Gyn specialists in basic care will most likely benefit from SIM-UT in addition to their clinical training regardless of the number of performed scans upfront in their clinical routine. The strengths of the study include the highly qualified expert group consisting of only certified DEGUM experts with total experience of 264,000 US scans. Data from the standardized hands-on test, which was collected to analyze learning progress and to measure the performance of the three groups, discriminated between the different US competence levels of the groups. Subsequently, the expert group performed significantly better, while the physician group outperformed the trial group in terms of accuracy until the trial group gained experience of 8 hours of SIM-UT. That indicates the realism of the simulator, the possibility of assessing different US competence levels with a standardized hands-on simulator test, and the feasibility of learning 2nd trimester scans by structured SIM-UT even with no to minimum experience. Simulation-based examinations for the assessment have already been shown to be a useful method [18 ]
[19 ]
[20 ]. If competence levels and skills can be assessed in a reliable way with US simulators, simulation-based tests could be an answer to long demanded accreditation and quality assurance for prenatal US [7 ]. Another strength of our study is the length of the intervention period, providing structured SIM-UT on a regular basis over six weeks, and the extent of the practiced examination protocol. Throughout the training period, the trial group recorded a significant decrease in TTC between each test. At the end of the training period, there were only three standard views that were accomplished significantly faster by the expert reference group. This crucial finding demonstrates that even very difficult standard views of extended protocols can be learned with SIM-UT. The positive feedback matches previous data about SIM-UT. SIM-UT is accepted among trainees in postgraduate training and is considered an essential part of their US education [24 ]. A limitation was the number of trial participants, which had to be kept limited, to allow enough time for training and skill progress testing. In detail, one simulator was available and shared with a parallel study with another group of 11 participants. Overall, 22 trainees each trained 2 hours per week on one simulator. Remaining time capacities were utilized to test skill progress and to evaluate the reference groups. As another limitation, we noticed a high rate of clinically properly obtained images that were not assigned to the matching standard plane. 268 views did not meet the simulator’s criteria whereas after expert panel review only 32/268 were classified as clinically insufficient. This discrepancy raises the question how the simulator evaluates the obtained images and when and in which situation the standard planes are rejected. In our experience, the tolerance should be adjusted for the acceptance of planes. Overall, simulators provide a safe learning space, with examinations being reproducible and cost-efficient [25 ]. In a standardized simulated setting trainees can focus on obtaining correct standard views to be able to detect fetal anomalies [26 ]. Individual different fetal and maternal physical characteristics can challenge sonographers and have not yet been implemented into the simulated examination conditions. Regardless of the latter, there is consensus regarding the efficacy and feasibility of SIM-UT in Ob/Gyn US [15 ]
[16 ]
[8 ]
[9 ]
[17 ]. The transferability of SIM-UT to real patients is part of successor trials. The quality of US examinations is significantly limited by the experience of the US operator, regardless of being a physician or technical sonographer [7 ]. Most importantly, while training US skills independent of mode of training (simulator vs. real patients), knowledge of US machines, technical pitfalls, and the knowledge behind image generation should be an inevitable part of all US teaching [27 ]. Residents should have the opportunity to train on US simulators, as errors on real patients can effectively be limited through SIM-UT [28 ]. Structured SIM-UT is a highly effective tool to learn basic and extended prenatal US. While enabling significant skill acquisition within short periods, it provides a safe and learner-centered atmosphere and can be conducted independently during medical curricula and residency training programs. In an age of virtual technology, physicians should benefit from these realistic simulations of diagnostic and therapeutic procedures, in addition to their daily clinical practice [28 ]. Overall, standardized hands-on tests on US simulators are a reliable tool to assess operator skills and assure quality standards of US examinations.
