Einsatz von erweiterten Realitäten (XR) in der Thoraxchirurgie

Zusammenfassung

Erweiterte Realitäten (XR) mit den Unterbegriffen Virtual Reality (VR), Augmented Reality (AR) und Mixed Reality (MR) beschreiben interaktive und immersive Technologien, welche die reelle Welt durch digitale Elemente ersetzen oder diese um solche nahtlos erweitern. Die XR bieten somit ein sehr breites mögliches Anwendungsspektrum in der Medizin. In der Chirurgie und insbesondere in der Thoraxchirurgie lassen sich XR-Technologien für Anwendungen der Behandlungsplanung, Navigation, Ausbildung und Patienteninformationen nutzbar machen. Solche Anwendungsfälle befinden sich zunehmend in der Erprobung und müssen hinsichtlich ihrer Wertigkeit evaluiert werden. Wir geben einen Überblick über den Status quo der technischen Entwicklung, aktuelle chirurgische Einsatzgebiete der XR und blicken in die Zukunft der medizinischen XR-Landschaft mit Integration von künstlicher Intelligenz (KI).

Abstract

Extended reality (XR) includes the sub-terms of virtual reality (VR), augmented reality (AR) and mixed reality (MR) and describes interactive and immersive technologies that replace the real world with digital elements or seamlessly extend it with such approaches. XR thus offers a very wide range of possible applications in medicine. In surgery, and thoracic surgery in particular, XR technologies can be harnessed for treatment planning, navigation, training, and patient information. Such applications are increasingly being tested and need to be evaluated. We provide an overview of the status quo of technical development, current surgical applications of XR, and look into the future of the medical XR landscape with integration of artificial intelligence (AI).

Schlüsselwörter

erweiterte Realität - virtuelle Realität - Navigation - künstliche Intelligenz - Simulation - chirurgische Lehre

Keywords

extended reality - virtual reality - augmented reality - mixed reality - surgical planning - image guided navigation

Publication History

Received: 17 April 2023

Accepted after revision: 29 June 2023

Article published online:
10 August 2023

• Literatur

• 1 Azuma RT. A survey of augmented reality. Presence: Teleoperators & Virtual Environments 1997; 6: 355-385
• 2 Kuhn S, Huettl F, Deutsch K. et al. Chirurgische Ausbildung im digitalen Zeitalter – Virtual Reality, Augmented Reality und Robotik im Medizinstudium. Zentralbl Chir 2021; 146: 37-43 DOI: 10.1055/a-1265-7259. (PMID: 33588501)
• 3 Alaker M, Wynn GR, Arulampalam T. Virtual reality training in laparoscopic surgery: a systematic review & meta-analysis. Int J Surg 2016; 29: 85-94 DOI: 10.1016/j.ijsu.2016.03.034. (PMID: 26992652)
• 4 Omlor A, Schwärzel L, Bewarder M. et al. Comparison of immersive and non-immersive virtual reality videos as substitute for in-hospital teaching during coronavirus lockdown: a survey with graduate medical students in Germany. Med Educ Online 2022; 27: 2101417 DOI: 10.1080/10872981.2022.2101417. (PMID: 35850619)
• 5 Barsom E, Graafland M, Schijven M. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 2016; 30: 4174-4183 DOI: 10.1007/s00464-016-4800-6. (PMID: 26905573)
• 6 Reinschluessel A, Muender T, Salzmann D. et al. Virtual Reality for Surgical Planning – Evaluation Based on Two Liver Tumor Resections. Front Surg 2022; 9: 821060 DOI: 10.3389/fsurg.2022.821060. (PMID: 35296126)
• 7 Uppot R, Laguna B, McCarthy C. et al. Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care. Radiology 2019; 291: 182210 DOI: 10.1148/radiol.2019182210. (PMID: 30990383)
• 8 Rahman R, Wood ME, Qian L. et al. Head-Mounted Display Use in Surgery: A Systematic Review. Surgical Innovation 2020; 27: 88-100 DOI: 10.1177/1553350619871787. (PMID: 31514682)
• 9 Torres K, Denisow-Pietrzyk M, Pietrzyk Ł. et al. Does simulation-based training facilitate the integration of human anatomy with surgery? A report of a novel Surgical Anatomy Course. Folia Morphol (Warsz) 2018; 77: 279-285 DOI: 10.5603/FM.a2017.0088. (PMID: 29064555)
• 10 Bakhuis W, Sadeghi AH, Moes I. et al. Essential Surgical Plan Modifications After Virtual Reality Planning in 50 Consecutive Segmentectomies. Ann Thorac Surg 2023; 115: 1247-1255 DOI: 10.1016/j.athoracsur.2022.08.037. (PMID: 36084694)
• 11 Marr B. What is extended reality technology? A simple explanation for anyone. Forbes; 2022. Accessed April 10, 2023 at: https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/?sh=6738782a7249
• 12 Marre P, Villet R. Anatomy theaters in the history and teaching of surgery. J Visc Surg 2020; 157 (Suppl. 02) S73-S76 DOI: 10.1016/j.jviscsurg.2020.03.005. (PMID: 32359884)
• 13 Kemp M. Style and non-style in anatomical illustration: From Renaissance Humanism to Henry Gray. J Anat 2010; 216: 192-208 DOI: 10.1111/j.1469-7580.2009.01181.x. (PMID: 20447244)
• 14 Azer S, Azer S. 3D Anatomy Models and Impact on Learning: A Review of the Quality of the Literature. Health Professions Education 2016; 2 DOI: 10.1016/j.hpe.2016.05.002.
• 15 Kockro R, Amaxopoulou C, Killeen T. et al. Stereoscopic neuroanatomy lectures using a three-dimensional virtual reality environment. Ann Anat 2015; 201: 91-98 DOI: 10.1016/j.aanat.2015.05.006. (PMID: 26245861)
• 16 Tang YM, Chau KY, Kwok APK. et al. A systematic review of immersive technology applications for medical practice and education – Trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ Res Rev 2022; 35: 100429 DOI: 10.1016/j.edurev.2021.100429.
• 17 Carpinello A, Vezzetti E, Ramieri G. et al. Evaluation of HMDs by QFD for Augmented Reality Applications in the Maxillofacial Surgery Domain. Appl Sci 2021; 11: 11053 DOI: 10.3390/app112211053.
• 18 Kolla S, Elgawly M, Gaughan J. et al. Medical Student Perception of a Virtual Reality Training Module for Anatomy Education. Med Sci Educ 2020; 30: 1201-1210 DOI: 10.1007/s40670-020-00993-2. (PMID: 34457783)
• 19 Ma M, Fallavollita P, Seelbach I. et al. Personalized augmented reality for anatomy education. Clin Anat 2016; 29: 446-453 DOI: 10.1002/ca.22675. (PMID: 26646315)
• 20 Nakai K, Terada S, Takahara A. et al. Anatomy education for medical students in a virtual reality workspace: A pilot study. Clin Anat 2022; 35: 40-44 DOI: 10.1002/ca.23783. (PMID: 34487367)
• 21 Feodorovici P, Bergedieck B, Sommer N. et al. Entwicklung und Implementierung eines Systems zur Virtual Reality (VR) gestützten Chirurgischen Lehre und Thoraxchirurgischen Weiterbildung am Universitätsklinikum Bonn. Zentralbl Chir 2022; 147 (Suppl. 01) S65 DOI: 10.1055/s-0042-1754232.
• 22 Sader J, Clavien C, Korris J. et al. Serious game training in medical education: potential to mitigate cognitive biases of healthcare professionals. Diagnosis (Berl) 2021; 8: 536-537 DOI: 10.1515/dx-2021-0004. (PMID: 33607705)
• 23 Graafland M, Schraagen JM, Schijven M. Systematic review of serious games for medical education and surgical skills training. Br J Surg 2012; 99: 1322-1330 DOI: 10.1002/bjs.8819. (PMID: 22961509)
• 24 Jensen K, Bjerrum F, Hansen HJ. et al. A new possibility in thoracoscopic virtual reality simulation training: development and testing of a novel virtual reality simulator for video-assisted thoracoscopic surgery lobectomy. Interact Cardiovasc Thorac Surg 2015; 21: 420-426 DOI: 10.1093/icvts/ivv183. (PMID: 26162895)
• 25 Spethmann S, Köhler F. Telemedizin bei chronischer Herzinsuffizienz – von klinischen Studien zur Regelversorgung. Internist (Berl) 2022; 63: 266-273 DOI: 10.1007/s00108-022-01268-1. (PMID: 35138432)
• 26 Kremers M. Teleradiologie und Telemedizin. MKG-Chirurg 2020; 13: 248-259
• 27 Ali S, AbdullahArmand TPT. et al. Metaverse in Healthcare Integrated with Explainable AI and Blockchain: Enabling Immersiveness, Ensuring Trust, and Providing Patient Data Security. Sensors 2023; 23 DOI: 10.3390/s23020565.
• 28 Fréchette E, Deslauriers J. Surgical Anatomy of the Bronchial Tree and Pulmonary Artery. Sem Thorac Cardiovasc Surg 2006; 18: 77-84 DOI: 10.1053/j.semtcvs.2006.06.002C. (PMID: 17157224)
• 29 Saji H, Okada M, Tsuboi M. et al. Segmentectomy versus lobectomy in small-sized peripheral non-small-cell lung cancer (JCOG0802/WJOG4607L): a multicentre, open-label, phase 3, randomised, controlled, non-inferiority trial. Lancet 2022; 399: 1607-1617 DOI: 10.1016/S0140-6736(21)02333-3. (PMID: 35461558)
• 30 Zhang L, Li M, Li Z. et al. Three-dimensional printing of navigational template in localization of pulmonary nodule: A pilot study. J Thorac Cardiovasc Surg 2017; 154: 2113-2119.e7 DOI: 10.1016/j.jtcvs.2017.08.065. (PMID: 29017792)
• 31 Kwok J, Lau R, Zhao ZR. et al. Multi-dimensional printing in thoracic surgery: Current and future applications. J Thorac Dis 2018; 10 (Suppl. 06) S756-S763 DOI: 10.21037/jtd.2018.02.91. (PMID: 29732197)
• 32 Xu W, Zhang X, Ke T. et al. 3D printing-assisted preoperative plan of pedicle screw placement for middle-upper thoracic trauma: a cohort study. BMC Musculoskelet Disord 2017; 18: 348 DOI: 10.1186/s12891-017-1703-1. (PMID: 28800768)
• 33 Bartella AK, Kamal M, Scholl I. et al. Virtual reality in preoperative imaging in maxillofacial surgery: implementation of “the next level”?. Br J Oral Maxillofac Surg 2019; 57: 644-648 DOI: 10.1016/j.bjoms.2019.02.014. (PMID: 31204187)
• 34 Dharmarajan H, Anderson J, Kim S. et al. Transition to a Virtual Multidisciplinary Tumor Board during the COVID-19 Pandemic: The University of Pittsburgh Experience. Head Neck 2020; 42: 1310-1316 DOI: 10.1002/hed.26195. (PMID: 32329958)
• 35 Blasi L, Bordonaro R, Serretta V. et al. Virtual Clinical and Precision Medicine Tumor Boards, a Cloud-based Platform-mediated Implementation of Multidisciplinary Reviews among Oncology Centers in the Covid-19 Era: an Observational Study Protocol. (Preprint). JMIR Research Protocols 2020; 10: e26220 DOI: 10.2196/26220. (PMID: 34387553)
• 36 Park CH, Han K, Hur J. et al. Comparative Effectiveness and Safety of Preoperative Lung Localization for Pulmonary Nodules: A Systematic Review and Meta-analysis. Chest 2017; 151: 316-328 DOI: 10.1016/j.chest.2016.09.017. (PMID: 27717643)
• 37 Fang HY, Chang KW, Chao YK. Hybrid operating room for the intraoperative CT-guided localization of pulmonary nodules. Ann Transl Med 2019; 7: 34 DOI: 10.21037/atm.2018.12.48. (PMID: 30854387)
• 38 Reisenauer J, Simoff MJ, Pritchett MA. et al. Ion: Technology and Techniques for Shape-sensing Robotic-assisted Bronchoscopy. Ann Thorac Surg 2022; 113: 308-315 DOI: 10.1016/j.athoracsur.2021.06.086. (PMID: 34370981)
• 39 Ng CS, Ong BH, Chao YK. et al. Use of Indocyanine Green Fluorescence Imaging in Thoracic and Esophageal Surgery. Ann Thorac Surg 2023; 115: 1068-1076 DOI: 10.1016/j.athoracsur.2022.06.061. (PMID: 36030832)
• 40 Hennig S, Jansen-Winkeln B, Köhler H. et al. Novel Intraoperative Imaging of Gastric Tube Perfusion during Oncologic Esophagectomy – A Pilot Study Comparing Hyperspectral Imaging (HSI) and Fluorescence Imaging (FI) with Indocyanine Green (ICG). Cancers (Basel) 2021; 14: 97 DOI: 10.3390/cancers14010097. (PMID: 35008261)
• 41 Lee YJ, van den Berg NS, Orosco RK. et al. A narrative review of fluorescence imaging in robotic-assisted surgery. Laparosc Surg 2021; 5: 31 DOI: 10.21037/ls-20-98. (PMID: 34549180)
• 42 Soyer P, Mosnier H, Choti MA. et al. Intraoperative and laparoscopic sonography of the liver. Eur Radiol 1997; 7: 1296-1302 DOI: 10.1007/s003300050292. (PMID: 9377518)
• 43 Khundam C, Vorachart V, Preeyawongsakul P. et al. A comparative study of interaction time and usability of using controllers and hand tracking in virtual reality training. Informatics 2021; 8: 60 DOI: 10.3390/informatics8030060.
• 44 Carter C, Graham R. Experimental comparison of manual and voice controls for the operation of in-vehicle systems. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2000; 44: 3–286-3–289 DOI: 10.1177/154193120004402016.
• 45 Greener R. What does CHATGPT mean for enterprise-grade XR?. XR Today; 2023. Accessed April 10, 2023 at: https://www.xrtoday.com/virtual-reality/what-does-chat-gpt-mean-for-enterprise-grade-xr/
• 46 Lo S, Fowers S, Darko K. et al. Participatory Development of a 3D Telemedicine system during Covid: the future of remote consultations. J Plast Reconstruct Aesthet Surg 2022; DOI: 10.1016/j.bjps.2022.10.012. (PMID: 36890078)
• 47 Microsoft Corporation, Microsoft Research. 3D telemedicine. Bringing specialized healthcare to rural and underserved communities with live 3D communication. 2023 Accessed April 10, 2023 at: https://www.microsoft.com/en-us/research/project/3d-telemedicine/