Simulation-Based Training of Ultrasound-Guided Procedures in Radiology – A Systematic Review

Abstract

Purpose To perform a systematic review of the effect of simulation-based training (SBT) of percutaneous abdominal and thoracic ultrasound-guided procedures and to assess the transfer of procedural competence to a clinical context.

Materials and Methods This systematic review was conducted in accordance with the PRISMA statement. Pubmed, Embase, Web of Science, and the Cochrane Library were searched for studies assessing procedural competence after SBT. Two authors independently reviewed all studies and extracted data. Risk of bias was assessed using the Cochrane tool for randomized studies (RoB) and non-randomized studies (ROBINS-I). Quality of evidence was assessed using the GRADE approach.

Results 42 studies were included. 6 were randomized controlled, 3 non-randomized controlled, and 33 non-randomized non-controlled. 26 studies examined US-guided abdominal procedures, 13 examined thoracic procedures, and 3 examined both. The results favored SBT compared to other educational interventions and found that training was superior to no training. Only two studies examined the transfer of procedural skills to a clinical context. All studies had a high or critical risk of bias. Thus, the quality of evidence for the effect of SBT on procedural competence was low, and evidence for its transfer to a clinical context was very low.

Conclusion The evidence supporting SBT of percutaneous abdominal and thoracic US-guided procedures remains insufficient due to methodological problems and a high risk of bias. Future studies should be randomized and single-blinded, use assessment tools supported by validity evidence, compare different educational strategies, and examine the transfer of skills to a clinical setting.

Zusammenfassung

Ziel Systematische Übersicht der Auswirkungen von simulationsbasiertem Training (SBT) bei perkutanen abdominalen und thorakalen ultraschallgestützten Verfahren und die Bewertung der Übertragung der Anwendungskompetenz in den klinischen Kontext.

Material und Methoden Diese systematische Überprüfung wurde nach der PRISMA-Checkliste durchgeführt. Pubmed, Embase, Web of Science und die Cochrane-Library wurden nach Studien durchsucht, die die Anwendungskompetenz nach SBT bewerteten. 2 Autoren überprüften unabhängig voneinander alle Studien und extrahierten die Daten. Das Risiko für Verzerrung wurde für randomisierte Studien mit dem Risk-of-Bias (RoB)-Tool nach Cochrane und für nicht randomisierte Studien (ROBINS-I) bewertet. Die Qualität der Evidenz wurde mit der GRADE-Methodik bewertet.

Ergebnisse 42 Studien wurden eingeschlossen; davon waren 6 randomisiert und kontrolliert, 3 nicht randomisiert und kontrolliert sowie 33 nicht randomisiert und nicht kontrolliert. 26 Studien untersuchten US-gestützte abdominale Verfahren, 13 untersuchten thorakale Verfahren und 3 untersuchten beide. Die Ergebnisse favorisierten SBT im Vergleich zu anderen Ausbildungsmethoden und es wurde festgestellt, dass eine Ausbildung besser war als keine. Nur 2 Studien untersuchten die Übertragung des verfahrenstechnischen Könnens in den klinischen Kontext. Alle Studien hatten ein hohes oder kritisches Bias-Risiko. Folglich war die Qualität der Evidenz bezüglich der Auswirkung der SBT auf die Anwendungskompetenz gering und für die Übertragung in den klinischen Kontext äußerst gering.

Schlussfolgerung Die Evidenz für SBT bei perkutanen abdominalen und thorakalen US-gestützten Verfahren ist aufgrund methodischer Probleme und eines hohen Bias-Risikos nach wie vor unzureichend. Zukünftige Studien sollten randomisiert und einfach-verblindet sein, Bewertungsmethoden verwenden, die durch Validitätsevidenz gestützt werden, unterschiedliche Ausbildungsmethoden vergleichen und den Transfer der Fähigkeiten in den klinischen Kontext untersuchen.

Key words

ultrasonography - simulation training - clinical competence - educational measurement - image-guided biopsy

• References

• 1 Gordon CE, Feller-Kopman D, Balk EM. et al. Pneumothorax following thoracentesis: a systematic review and meta-analysis. Arch Intern Med 2010; 170: 332-339
• 2 Meyer JE, Smith DN, Lester SC. et al. Large-core needle biopsy of nonpalpable breast lesions. JAMA 1999; 281: 1638-1641
• 3 Nazeer SR, Dewbre H, Miller AH. Ultrasound-assisted paracentesis performed by emergency physicians vs the traditional technique: a prospective, randomized study. The American Journal of Emergency Medicine; Philadelphia 2005; 23: 363-367
• 4 Tsang TSM, Enriquez-Sarano M, Freeman WK. et al. Consecutive 1127 therapeutic echocardiographically guided pericardiocenteses: clinical profile, practice patterns, and outcomes spanning 21 years. Mayo Clin Proc 2002; 77: 429-436
• 5 Dietrich CF, Lorentzen T, Appelbaum L. et al. EFSUMB Guidelines on Interventional Ultrasound (INVUS), Part III – Abdominal Treatment Procedures (Long Version). Ultraschall in Med 2016; 37: E1-E32
• 6 Lorentzen T, Nolsøe CP, Ewertsen C. et al. EFSUMB Guidelines on Interventional Ultrasound (INVUS), Part I – General Aspects (long Version). Ultraschall in Med 2015; 36: E1-E14
• 7 Sidhu PS, Brabrand K, Cantisani V. et al. EFSUMB Guidelines on Interventional Ultrasound (INVUS), Part II – Diagnostic Ultrasound-Guided Interventional Procedures (Long Version). Ultraschall in Med 2015; 36: E15-E35
• 8 ESR European Training Curriculum Level I – II (2018).pdf | myESR [cited 2018]. Available at: https://www.myesr.org/media/2838
• 9 European Society of Radiology (ESR). Organisation and practice of radiological ultrasound in Europe: a survey by the ESR Working Group on Ultrasound. Insights into Imaging 2013; 4: 401-407
• 10 European Society of Radiology (ESR). International Summit 2014: Organisation of clinical ultrasound in the world. Insights into Imaging 2014; 5: 641-644
• 11 Reznick RK, MacRae H. Teaching Surgical Skills — Changes in the Wind. New England Journal of Medicine 2006; 355: 2664-2669
• 12 Nguyen S, Ferland N, Beaudoin S. et al. Influence of trainee involvement on procedural characteristics for linear endobronchial ultrasound: Endobronchial ultrasound and trainees. Thoracic Cancer 2017; 8: 517-522
• 13 Stather DR, MacEachern P, Chee A. et al. Trainee Impact on Procedural Complications: An Analysis of 967 Consecutive Flexible Bronchoscopy Procedures in an Interventional Pulmonology Practice. Respiration 2013; 85: 422-428
• 14 Young JQ, Ranji SR, Wachter RM. et al. ‘July effect’: impact of the academic year-end changeover on patient outcomes: a systematic review. Ann Intern Med 2011; 155: 309-315
• 15 Grantcharov TP, Kristiansen VB, Bendix J. et al. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. BJS 2004; 91: 146-150
• 16 Thomsen ASS, Bach-Holm D, Kjærbo H. et al. Operating Room Performance Improves after Proficiency-Based Virtual Reality Cataract Surgery Training. Ophthalmology 2017; 124: 524-531
• 17 Tolsgaard MG, Ringsted C, Dreisler E. et al. Sustained effect of simulation-based ultrasound training on clinical performance: a randomized trial. Ultrasound Obstet Gynecol 2015; 46: 312-318
• 18 Griswold-Theodorson S, Ponnuru S, Dong C. et al. Beyond the simulation laboratory: a realist synthesis review of clinical outcomes of simulation-based mastery learning. Acad Med 2015; 90: 1553-1560
• 19 World Health Organization. Transforming and scaling up health professionals’ education and training: World Health Organization guidelines. 2013
• 20 Nayahangan LJ, Nielsen KR, Albrecht-Beste E. et al. Determining procedures for simulation-based training in radiology: a nationwide needs assessment. Eur Radiol 2018; 28: 2319-2327
• 21 Acton RD, Chipman JG, Lunden M. et al. Unanticipated Teaching Demands Rise with Simulation Training: Strategies for Managing Faculty Workload. Journal of Surgical Education 2015; 72: 522-529
• 22 Dyre L, Tolsgaard MG. The gap in transfer research. Medical Education 2018; 52: 580-582
• 23 Zendejas B, Wang AT, Brydges R. et al. Cost: The missing outcome in simulation-based medical education research: A systematic review. Surgery 2013; 153: 160-176
• 24 Moher D, Liberati A, Tetzlaff J. et al. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine 2009; 6: e1000097
• 25 Higgins JPT, Altman DG, Gøtzsche PC. et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ: British Medical Journal (Online); London 2011; 343: d5928
• 26 Sterne JA, Hernán MA, Reeves BC. et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355: i4919 . doi:10.1136/bmj.i4919
• 27 Guyatt G, Oxman AD, Akl EA. et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011; 64: 383-394
• 28 Alba GA, Kelmenson DA, Noble VE. et al. Faculty staff-guided versus self-guided ultrasound training for internal medicine residents. Medical Education 2013; 47: 1099-1108
• 29 Mourad M, Ranji S, Sliwka D. A Randomized Controlled Trial of the Impact of a Teaching Procedure Service on the Training of Internal Medicine Residents. Journal of Graduate Medical Education 2012; 4: 170-175
• 30 Hassard MK, McCurdy LI, Williams JCA. et al. Training module to teach ultrasound-guided breast biopsy skills to residents improve accuracy. Canadian Association of Radiologists Journal 2003; 54: 155
• 31 Filippou P, Odisho A, Ramaswamy K. et al. Using an abdominal phantom to teach urology residents ultrasound-guided percutaneous needle placement. Int Braz J Urol 2016; 42: 717-726
• 32 Francesconi M, Freschi C, Sinceri S. et al. New training methods based on mixed reality for interventional ultrasound: Design and validation. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2015
• 33 Fulton N, Buethe J, Gollamudi J. et al. Simulation-Based Training May Improve Resident Skill in Ultrasound-Guided Biopsy. American Journal of Roentgenology 2016; 207: 1329-1333
• 34 Duncan DR, Morgenthaler TI, Ryu JH. et al. Reducing Iatrogenic Risk in Thoracentesis: Establishing Best Practice Via Experiential Training in a Zero-Risk Environment. Chest 2009; 135: 1315-1320
• 35 Dawoud D, Lyndon W, Mrug S. et al. Impact of Ultrasound-Guided Kidney Biopsy Simulation on Trainee Confidence and Biopsy Outcomes. American Journal of Nephrology 2012; 36: 570-574
• 36 Ristolainen A, Ross P, Gavšin J. et al. Economically affordable anatomical kidney phantom with calyxes for puncture and drainage training in interventional urology and radiology. Acta Radiol Short Rep 2014; 3: 2047981614534231 . doi:10.1177/2047981614534231. eCollection 2014 Jun.
• 37 Vetrugno L, Volpicelli G, Barbariol F. et al. Phantom model and scoring system to assess ability in ultrasound-guided chest drain positioning. Crit Ultrasound J 2016; 8: 1 . doi:10.1186/s13089-016-0038-8. Epub 2016 Feb 18
• 38 Keddis MT, Cullen MW, Reed DA. et al. Effectiveness of an Ultrasound Training Module for Internal Medicine Residents. BMC Med Educ 2011; 11: 75
• 39 Campo dell’ Orto M, Hempel D, Starzetz A. et al. Assessment of a Low-Cost Ultrasound Pericardiocentesis Model. Emerg Med Int 2013; 2013: 376415 . doi:10.1155/2013/376415. Epub 2013 Oct 29
• 40 Daly R, Planas JH, Edens MA. Adapting Gel Wax into an Ultrasound-Guided Pericardiocentesis Model at Low Cost. West J Emerg Med 2017; 18: 114-116
• 41 Sullivan A, Khait L, Favot M. A Novel Low-Cost Ultrasound-Guided Pericardiocentesis Simulation Model: Demonstration of Feasibility. Journal of Ultrasound in Medicine 2018; 37: 493-500
• 42 Gresens AA, Britt RC, Feliberti EC. et al. Ultrasound-guided breast biopsy for surgical residents: evaluation of a phantom model. J Surg Educ 2012; 69: 411-415
• 43 Harvey JA, Moran RE, Hamer MM. et al. Evaluation of a turkey-breast phantom for teaching freehand, US-guided core-needle breast biopsy. Academic Radiology 1997; 4: 565-569
• 44 Hoover SJ, Berry MP, Rossick L. et al. Ultrasound-Guided Breast Biopsy Curriculum for Surgical Residents. Surg Innov 2008; 15: 52-58
• 45 Law MT, Bennett IC. Structured Ultrasonography Workshop for Breast Surgeons: Is It An Effective Training Tool?. World Journal of Surgery; Lupsingen 2010; 34: 549-554
• 46 Meng K, Lipson JA. Utilizing a PACS-integrated Ultrasound-guided Breast Biopsy Simulation Exercise to Reinforce the ACR Practice Guideline for Ultrasound-Guided Percutaneous Breast Interventional Procedures During Radiology Residency. Academic Radiology 2011; 18: 1324-1328
• 47 Roark AA, Ebuoma LO, Ortiz-Perez T. et al. Impact of Simulation-Based Training on Radiology Trainee Education in Ultrasound-Guided Breast Biopsies. J Am Coll Radiol 2018; 15: 1458-1463 . doi:10.1016/j.jacr.2017.09.016. Epub 2017 Dec 6
• 48 Sutcliffe J, Hardman RL, Dornbluth NC. et al. A Novel Technique for Teaching Challenging Ultrasound-Guided Breast Procedures to Radiology Residents. Journal of Ultrasound in Medicine 2013; 32: 1845-1854
• 49 Barsuk JH, Cohen ER, Vozenilek JA. et al. Simulation-Based Education with Mastery Learning Improves Paracentesis Skills. J Grad Med Educ 2012; 4: 23-27
• 50 Fortmeier D, Mastmeyer A, Schröder J. et al. A Virtual Reality System for PTCD Simulation Using Direct Visuo-Haptic Rendering of Partially Segmented Image Data. IEEE J Biomed Health Inform 2016; 20: 355-366
• 51 Mohr A, Jung EM, Stroszczynski C. et al. New economic training model for installing ultrasound-guided drainages. Z Gastroenterol 2014; 52: 1257-1262
• 52 Johnson SJ, Hunt CM, Woolnough HM. et al. Virtual reality, ultrasound-guided liver biopsy simulator: development and performance discrimination. Br J Radiol 2012; 85: 555-561
• 53 Nicotra JJ, Gay SB, Wallace KK. et al. Evaluation of a Breast Biopsy Phantom for Learning Freehand Ultrasound-Guided Biopsy of the Liver. Academic Radiology 1994; 1: 385-387
• 54 Pacioni A, Carbone M, Freschi C. et al. Patient-specific ultrasound liver phantom: materials and fabrication method. Int J CARS 2015; 10: 1065-1075
• 55 Sekhar A, Sun MR, Siewert B. A Tissue Phantom Model for Training Residents in Ultrasound-guided Liver Biopsy. Academic Radiology 2014; 21: 902-908
• 56 Wu W, Xue Y, Wang D. et al. A simulator for percutaneous hepatic microwave thermal ablation under ultrasound guidance. International Journal of Hyperthermia 2014; 30: 429-437
• 57 Meek MEM, Meek JC, Hollowoa B. et al. Lightly Embalmed Cadavers as a Training Tool for Ultrasound-Guided Procedures Commonly Used in Interventional Radiology. Acad Radiol 2018; 25: 1503-1509 . doi:10.1016/j.acra.2018.05.019. Epub 2018 Jul 17
• 58 Hunt A, Ristolainen A, Ross P. et al. Low cost anatomically realistic renal biopsy phantoms for interventional radiology trainees. European Journal of Radiology 2013; 82: 594-600
• 59 Oliver SW, Patel RK, Ali KA. et al. Teaching percutaneous renal biopsy using unfixed human cadavers. BMC Nephrol 2015; 16: 209 . doi:10.1186/s12882-015-0210-6
• 60 Sharma SG, Arthur JM, Bonsib SM. et al. An integrated pathology and ultrasonography-based simulation for training in performing kidney biopsy. Clin Nephrol 2018; 89: 214-222
• 61 Woywodt A, How T, Schulz M. A purpose-built simulator for percutaneous ultrasound-guided renal biopsy. Clin Nephrol 2013; 79: 241-245
• 62 Qiu Z, Yang Y, Zhang Y. et al. Modified biological training model for percutaneous renal surgery with ultrasound and fluroscopy guidance. Chin Med J 2011; 124: 1286-1289
• 63 Zhang Y, Ou T, Jia J. et al. Novel Biologic Model for Percutaneous Renal Surgery Learning and Training in the Laboratory. Urology 2008; 72: 513-516
• 64 Chan WY, Qin J, Chui YP. et al. A Serious Game for Learning Ultrasound-Guided Needle Placement Skills. IEEE Transactions on Information Technology in Biomedicine 2012; 16: 1032-1042
• 65 Magee D, Zhu Y, Ratnalingam R. et al. An augmented reality simulator for ultrasound guided needle placement training. Medical and Biological Engineering and Computing; Heidelberg 2007; 45: 957-967
• 66 Mendiratta-Lala M, Williams T, de Quadros N. et al. The Use of a Simulation Center to Improve Resident Proficiency in Performing Ultrasound-Guided Procedures. Academic Radiology 2010; 17: 535-540
• 67 Ni D, Chan WY, Qin J. et al. A Virtual Reality Simulator for Ultrasound-Guided Biopsy Training. IEEE Computer Graphics and Applications 2011; 31: 36-48
• 68 Sevak S, Lurvey B, Woodfin AA. et al. Solid, Cystic, and Tubular: Novice Ultrasound Skills Training Using a Versatile, Affordable Practice Model. J Surg Educ 2018; 75: 1403-1409 . doi:10.1016/j.jsurg.2018.02.010. Epub 2018 Apr 9
• 69 Tabriz DM, Street M, Pilgram TK. et al. Objective assessment of operator performance during ultrasound-guided procedures. Int J CARS 2011; 6: 641-652
• 70 Norman G. Data dredging, salami-slicing, and other successful strategies to ensure rejection: twelve tips on how to not get your paper published. Adv in Health Sci Educ 2014; 19: 1-5
• 71 Cook DA, Beckman TJ. Reflections on experimental research in medical education. Adv in Health Sci Educ 2010; 15: 455-464
• 72 Downing SM, Yudkowsky R. Assessment in Health Professions Education. New York: Routledge; 2009 1 edition.
• 73 Cook DA, Hatala R, Brydges R. et al. Technology-Enhanced Simulation for Health Professions Education: A Systematic Review and Meta-analysis. JAMA 2011; 306: 978-988
• 74 Norman G, Dore K, Grierson L. The minimal relationship between simulation fidelity and transfer of learning. Medical Education 2012; 46: 636-647
• 75 Kohls-Gatzoulis JA, Regehr G, Hutchison C. Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg 2004; 47: 277-283
• 76 Eva KW, Regehr G. Self-Assessment in the Health Professions: A Reformulation and Research Agenda. Academic Medicine 2005; 80: S46-S54
• 77 Davis DA, Mazmanian PE, Fordis M. et al. Accuracy of Physician Self-assessment Compared With Observed Measures of Competence: A Systematic Review. JAMA 2006; 296: 1094-1102
• 78 Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65: S63-67
• 79 Borgersen NJ, Naur TMH, Sørensen SMD. et al. Gathering Validity Evidence for Surgical Simulation: A Systematic Review. Ann Surg 2018; 267: 1063-1068
• 80 Østergaard ML, Nielsen KR, Albrecht-Beste E. et al. Development of a reliable simulation-based test for diagnostic abdominal ultrasound with a pass/fail standard usable for mastery learning. Eur Radiol 2018; 28: 51-57
• 81 Savran MM, Sørensen SMD, Konge L. et al. Training and Assessment of Hysteroscopic Skills: A Systematic Review. Journal of Surgical Education 2016; 73: 906-918
• 82 Konge L, Clementsen PF, Ringsted C. et al. Simulator training for endobronchial ultrasound: a randomised controlled trial. European Respiratory Journal 2015; 46: 1140-1149