Augmented Reality in der Kopf-Hals-Chirurgie

Zusammenfassung

Augmented Reality (AR) ist eine aufstrebende Technologie, die Operateure beim räumlichen Denken während klinischer Eingriffe unterstützt. Zu den faszinierendsten Anwendungen gehören Visualisierungen von anatomischen Strukturen, Schnittbildern oder Zugangswegen, die in Kombination mit der realen Umgebung direkt am Patienten angezeigt werden können. Das Ziel dieser narrativen Übersicht besteht darin, den Stand der Technik und die zukünftigen Richtungen der AR in der Kopf-Hals-Chirurgie zu skizzieren. Besonderes Augenmerk liegt dabei auf der Darlegung des Potenzials und der noch zu lösenden Schwierigkeiten dieser neuen Technik.

Abstract

Augmented Reality (AR) is an emerging technology that supports surgeons in spatial reasoning during clinical procedures. The most fascinating applications include visualizations of anatomical structures, cross-sectional images or surgical approaches, which can be displayed directly on the patient in combination with the real world. The aim of this narrative review is to outline the state of the art and future directions of AR in head and neck surgery. Particular attention is paid to explaining the potential and the difficulties that still need to be solved with this new technology.

Schlüsselwörter

Augmented Reality - Image Guidance - Navigation - virtuell - dreidimensional

Keywords

Augmented Reality - Image Guidance - Navigation - virtual - three dimensional

Publikationsverlauf

Eingereicht: 28. September 2023

Angenommen nach Revision: 06. November 2024

Artikel online veröffentlicht:
03. Dezember 2024

• Literatur

• 1 Ayoub A, Pulijala Y. The application of virtual reality and augmented reality in Oral & Maxillofacial Surgery. BMC Oral Health 2019; 19: 238
• 2 Heilig ML. Sensorama simulator. US PAT 3,050,870. 1962
• 3 Citardi MJ, Agbetoba A, Bigcas JL. et al. Augmented reality for endoscopic sinus surgery with surgical navigation: a cadaver study. International forum of allergy & rhinology 2016; 523-528
• 4 Ramakrishnan VR, Orlandi RR, Citardi MJ. et al. The use of image-guided surgery in endoscopic sinus surgery: an evidence-based review with recommendations. International forum of allergy & rhinology 2013; 236-241
• 5 Chand M, Keller DS, Devoto L. et al. Furthering precision in sentinel node navigational surgery for Oral Cancer: a novel triple targeting system. Journal of Fluorescence 2018; 28: 483-486
• 6 Marmulla R, Hoppe H, Mühling J. et al. New augmented reality concepts for craniofacial surgical procedures. Plastic and Reconstructive Surgery 2005; 115: 1124-1128
• 7 Li L, Yang J, Chu Y. et al. A novel augmented reality navigation system for endoscopic sinus and skull base surgery: a feasibility study. PLoS One 2016; 11: e0146996
• 8 Suzuki N, Hattori A, Iimura J. et al. Development of AR Surgical Navigation Systems for Multiple Surgical Regions. MMVR 2014; 404-408
• 9 Salisbury JP, Keshav NU, Sossong AD. et al. Concussion assessment with smartglasses: validation study of balance measurement toward a lightweight, multimodal, field-ready platform. JMIR mHealth and uHealth 2018; 6: e8478
• 10 Scherl C, Männle D, Rotter N. et al. Augmented reality during parotid surgery: real-life evaluation of voice control of a head mounted display. Eur Arch Otorhinolaryngol 2023; 280: 2043-2049
• 11 Sahovaler A, Chan HH, Gualtieri T. et al. Augmented reality and intraoperative navigation in sinonasal malignancies: A preclinical study. Frontiers in Oncology 2021; 11: 723509
• 12 Khamene A, Acker FW, Vogt S. et al. An augmented reality system for MRI-guided needle biopsies. Medicine Meets Virtual Reality 11. IOS Press; 2003. 11.
• 13 McJunkin JL, Jiramongkolchai P, Chung W. et al. Development of a mixed reality platform for lateral skull base anatomy. Otology & Neurotology: Official Publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology 2018; 39: e1137
• 14 Meola A, Cutolo F, Carbone M. et al. Augmented reality in neurosurgery: a systematic review. Neurosurgical review 2017; 40: 537-548
• 15 Scherl C, Stratemeier J, Karle C. et al. Augmented reality with HoloLens in parotid surgery: how to assess and to improve accuracy. European Archives of Oto-Rhino-Laryngology 2021; 278: 2473-2483
• 16 Wong K, Yee HM, Xavier BA. et al. Applications of augmented reality in otolaryngology: a systematic review. Otolaryngology–Head and Neck Surgery 2018; 159: 956-967
• 17 Chen JX, Yu SE, Ding AS. et al. Augmented reality in otology/neurotology: a scoping review with implications for practice and education. The Laryngoscope 2023; 133: 1786-1795
• 18 Colombo E, Fick T, Esposito G. et al. Segmentation techniques of brain arteriovenous malformations for 3D visualization: a systematic review. Radiol Med 2022; 127: 1333-1341
• 19 Ke J, Lv Y, Ma F. et al. Deep learning-based approach for the automatic segmentation of adult and pediatric temporal bone computed tomography images. Quant Imaging Med Surg 2023; 13: 1577-1591
• 20 Andrews CM, Henry AB, Soriano IM. et al. Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices. IEEE J Transl Eng Health Med 2021; 9: 4900214
• 21 McJunkin JL, Jiramongkolchai P, Chung W. et al. Development of a Mixed Reality Platform for Lateral Skull Base Anatomy. Otol Neurotol 2018; 39: e1137-e1142
• 22 Li Y, Chen X, Wang N. et al. A wearable mixed-reality holographic computer for guiding external ventricular drain insertion at the bedside. J Neurosurg 2018; 1-8
• 23 Frantz T, Jansen B, Duerinck J. et al. Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation. Healthc Technol Lett 2018; 5: 221-225
• 24 Condino S, Sannino S, Cutolo F. et al. Single feature constrained manual registration method for Augmented Reality applications in gynecological laparoscopic interventions. Annu Int Conf IEEE Eng Med Biol Soc 2022; 2022: 566-571
• 25 Morita S, Suzuki K, Yamamoto T. et al. Out-of-Plane Needle Placements Using 3D Augmented Reality Protractor on Smartphone: An Experimental Phantom Study. Cardiovasc Intervent Radiol 2023; 46: 675-679
• 26 Jiang T, Yu D, Wang Y. et al. HoloLens-Based Vascular Localization System: Precision Evaluation Study With a Three-Dimensional Printed Model. J Med Internet Res 2020; 22: e16852
• 27 Müller F, Roner S, Liebmann F. et al. Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging. Spine J 2020; 20: 621-628
• 28 Meulstee JW, Nijsink J, Schreurs R. et al. Toward Holographic-Guided Surgery. Surg Innov 2019; 26: 86-94
• 29 Hogan J. Augmedics’ xvision Spine system. Zugriff am 21. Mai 2024 unter: https://www.accessdata.fda.gov/cdrh_docs/pdf19/K190929.pdf
• 30 Chien JC, Tsai Y-R, Wu CT. et al. HoloLens-based AR system with a robust point set registration algorithm. Sensors 2019; 19: 3555
• 31 Gsaxner C, Wallner J, Chen X. et al. Facial model collection for medical augmented reality in oncologic cranio-maxillofacial surgery. Scientific data 2019; 6: 310
• 32 Kwon JG, Hong DW, Choi JW. Clinical Applications of Augmented Reality Technology in Microsurgical Planning of Head and Neck Reconstruction. J Craniofac Surg 2022; 33: 863-866
• 33 Caversaccio M, Langlotz F, Nolte LP. et al. Impact of a self-developed planning and self-constructed navigation system on skull base surgery: 10 years experience. Acta Otolaryngol 2007; 127: 403-407
• 34 Barber SR, Wong K, Kanumuri V. et al. Augmented reality, surgical navigation, and 3D printing for transcanal endoscopic approach to the petrous apex. OTO open 2018; 2: 2473974X18804492
• 35 Tel A, Zeppieri M, Robiony M. et al. Exploring Deep Cervical Compartments in Head and Neck Surgical Oncology through Augmented Reality Vision: A Proof of Concept. J Clin Med 2023; 12
• 36 Goto Y, Kawaguchi A, Inoue Y. et al. Efficacy of a Novel Augmented Reality Navigation System Using 3D Computer Graphic Modeling in Endoscopic Transsphenoidal Surgery for Sellar and Parasellar Tumors. Cancers (Basel) 2023; 15
• 37 Bopp MHA, Saß B, Pojskić M. et al. Use of Neuronavigation and Augmented Reality in Transsphenoidal Pituitary Adenoma Surgery. J Clin Med 2022; 11
• 38 Dixon BJ, Daly MJ, Chan H. et al. Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery. The Laryngoscope 2014; 124: 853-859
• 39 Linxweiler M, Pillong L, Kopanja D. et al. Augmented reality-enhanced navigation in endoscopic sinus surgery: A prospective, randomized, controlled clinical trial. Laryngoscope Investigative Otolaryngology 2020; 5: 621-629
• 40 Necker FN, Chang M, Leuze C. et al. Virtual Resection Specimen Interaction Using Augmented Reality Holograms to Guide Margin Communication and Flap Sizing. Otolaryngology – Head and Neck Surgery 2023;
• 41 Pratt P, Ives M, Lawton G. et al. Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels. European radiology experimental 2018; 2: 1-7
• 42 Gorpas D, Phipps J, Bec J. et al. Autofluorescence lifetime augmented reality as a means for real-time robotic surgery guidance in human patients. Scientific reports 2019; 9: 1187
• 43 Liu WP, Richmon JD, Sorger JM. et al. Augmented reality and cone beam CT guidance for transoral robotic surgery. Journal of robotic surgery 2015; 9: 223-233
• 44 Agbetoba A, Luong A, Siow JK. et al. Educational utility of advanced three-dimensional virtual imaging in evaluating the anatomical configuration of the frontal recess. International forum of allergy & rhinology 2017; 07 (02) 143-148
• 45 Küçük S, Kapakin S, Göktaş Y. Learning anatomy via mobile augmented reality: Effects on achievement and cognitive load. Anat Sci Educ 2016; 9: 411-421
• 46 Bong JH, Song Hj, Oh Y. et al. Endoscopic navigation system with extended field of view using augmented reality technology. The International Journal of Medical Robotics and Computer Assisted Surgery 2018; 14: e1886
• 47 Chen X, Xu L, Wang Y. et al. Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. Journal of biomedical informatics 2015; 55: 124-131
• 48 Pollefeys M. Microsoft HoloLens facilitates computer vision research by providing access to raw image sensor streams with Research Mode. Zugriff am 21. Mai 2024 unter: https://www.microsoft.com/en-us/research/blog/microsoft-hololens-facilitates-computer-vision-research-by-providing-access-to-raw-image-sensor-streams-with-research-mode/
• 49 de Oliveira ME, Debarba HG, Lädermann A. et al. A hand-eye calibration method for augmented reality applied to computer-assisted orthopedic surgery. Int J Med Robot 2019; 15: e1969
• 50 Ahmed OH, Marcus S, Lebowitz RA. et al. Evolution in visualization for sinus and skull base surgery: from headlight to endoscope. Otolaryngologic Clinics of North America 2017; 50: 505-519
• 51 Tzelnick S, Rampinelli V, Sahovaler A. et al. Skull-Base Surgery – A Narrative Review on Current Approaches and Future Developments in Surgical Navigation. Journal of Clinical Medicine 2023; 12: 2706
• 52 Marroquin R, Lalande A, Hussain R. et al. Augmented reality of the middle ear combining otoendoscopy and temporal bone computed tomography. Otology & Neurotology 2018; 39: 931-939
• 53 Creighton FX, Unberath M, Song T. et al. Early feasibility studies of augmented reality navigation for lateral skull base surgery. Otology & Neurotology 2020; 41: 883-888
• 54 Liu WP, Azizian M, Sorger J. et al. Cadaveric feasibility study of da vinci si–assisted cochlear implant with augmented visual navigation for otologic surgery. JAMA Otolaryngology – Head & Neck Surgery 2014; 140: 208-214
• 55 Szewczyk M, Golusinski W, Pazdrowski J. et al. Positive fresh frozen section margins as an adverse independent prognostic factor for local recurrence in oral cancer patients. The Laryngoscope 2018; 128: 1093-1098
• 56 Kreeft AM, Smeele LE, Rasch CR. et al. Preoperative imaging and surgical margins in maxillectomy patients. Head & neck 2012; 34: 1652-1656
• 57 Chan HH, Sahovaler A, Daly MJ. et al. Projected cutting guides using an augmented reality system to improve surgical margins in maxillectomies: A preclinical study. Oral Oncology 2022; 127: 105775
• 58 Seetohul J, Shafiee M, Sirlantzis K. Augmented Reality (AR) for Surgical Robotic and Autonomous Systems: State of the Art, Challenges, and Solutions. Sensors (Basel) 2023; 23
• 59 Iqbal H, Tatti F, Rodriguez YBF. Augmented reality in robotic assisted orthopaedic surgery: A pilot study. J Biomed Inform 2021; 120: 103841
• 60 Dedhia RC, Lord CA, Pinheiro-Neto CD. et al. Endoscopic endonasal pituitary surgery: impact of surgical education on operation length and patient morbidity. Journal of Neurological Surgery Part B: Skull Base 2012; 405-409
• 61 Dombrowski T, Dazert S, Volkenstein S. Strategies of Digitized Learning. Laryngorhinootologie 2019; 98: S197-s219
• 62 Offergeld C, von Schnakenburg P, Knopf A. et al. Innovation trifft Tradition: Virtual Reality (VR) in der HNO-Lehre. Laryngo-Rhino-Otologie 2022; 101: 295-297
• 63 Botden SM, Buzink SN, Schijven MP. et al. Augmented versus virtual reality laparoscopic simulation: what is the difference? A comparison of the ProMIS augmented reality laparoscopic simulator versus LapSim virtual reality laparoscopic simulator. World journal of surgery 2007; 31: 764-772
• 64 James J, Irace AL, Gudis DA. et al. Simulation training in endoscopic skull base surgery: A scoping review. World Journal of Otorhinolaryngology – Head and Neck Surgery 2022; 8: 73-81
• 65 Kovoor JG, Gupta AK, Gladman MA. Validity and effectiveness of augmented reality in surgical education: a systematic review. Surgery 2021; 170: 88-98
• 66 Yong M, Pauwels J, Kozak FK. et al. Application of augmented reality to surgical practice: A pilot study using the ODG R7 Smartglasses. Clinical Otolaryngology 2020; 45: 130-134
• 67 Vera AM, Russo M, Mohsin A. et al. Augmented reality telementoring (ART) platform: a randomized controlled trial to assess the efficacy of a new surgical education technology. Surg Endosc 2014; 28: 3467-3472
• 68 El-Sabawi B, Magee III W. The evolution of surgical telementoring: current applications and future directions. Annals of translational medicine 2016; 4
• 69 Baumhauer M, Feuerstein M, Meinzer HP. et al. Navigation in endoscopic soft tissue surgery: perspectives and limitations. Journal of endourology 2008; 22: 751-766
• 70 Mangano FG, Admakin O, Lerner H. et al. Artificial intelligence and augmented reality for guided implant surgery planning: A proof of concept. J Dent 2023; 133: 104485
• 71 Chan HH, Haerle SK, Daly MJ. et al. An integrated augmented reality surgical navigation platform using multi-modality imaging for guidance. PLoS One 2021; 16: e0250558
• 72 Tabrizi LB, Mahvash M. Augmented reality–guided neurosurgery: accuracy and intraoperative application of an image projection technique. Journal of neurosurgery 2015; 123: 206-211
• 73 Luzon JA, Stimec BV, Bakka AO. et al. Value of the surgeon’s sightline on hologram registration and targeting in mixed reality. International journal of computer assisted radiology and surgery 2020; 15: 2027-2039
• 74 Lee S, Hu X, Hua H. Effects of optical combiner and IPD change for convergence on near-field depth perception in an optical see-through HMD. IEEE transactions on visualization and computer graphics 2015; 22: 1540-1554
• 75 Lai M, Skyrman S, Shan C. et al. Fusion of augmented reality imaging with the endoscopic view for endonasal skull base surgery; a novel application for surgical navigation based on intraoperative cone beam computed tomography and optical tracking. Plos one 2020; 15: e0227312
• 76 Chu Y, Yang J, Ma S. et al. Registration and fusion quantification of augmented reality based nasal endoscopic surgery. Medical image analysis 2017; 42: 241-256
• 77 Ruggiero F, Cercenelli L, Emiliani N. et al. Preclinical Application of Augmented Reality in Pediatric Craniofacial Surgery: An Accuracy Study. Journal of Clinical Medicine 2023; 12: 2693
• 78 Jiang T, Zhu M, Chai G. et al. Precision of a novel craniofacial surgical navigation system based on augmented reality using an occlusal splint as a registration strategy. Scientific reports 2019; 9: 501
• 79 Prasad K, Miller A, Sharif K. et al. Augmented-reality surgery to guide head and neck cancer re-resection: a feasibility and accuracy study. Annals of surgical oncology 2023; 1-7
• 80 Dixon BJ, Daly MJ, Chan HH. et al. Inattentional blindness increased with augmented reality surgical navigation. American journal of rhinology & allergy 2014; 28: 433-437