Augmented Reality und Virtual Reality im Operationssaal – Status Quo und Quo vadis

Zusammenfassung

Einleitung Virtual Reality (VR) ist die künstlich simulierte Umgebung, mit der eine Interaktion möglich ist. Die Augmented Reality (AR) hingegen überlagert reale und künstliche Informationen. Beide Verfahren sind im Klinikalltag bereits vielerorts integriert und werden zunehmend auch für den chirurgischen Sektor beworben. Welche Möglichkeiten die vielfältigen Anwendungsmöglichkeiten bieten, ist dabei vielen Ärzten unklar.

Ziel Erfassung der aktuellen und zukünftigen Verwendung von AR und VR im perioperativen Umfeld sowie die Veranschaulichung der zu erwartenden Verbesserungen und Probleme durch diese neuen Technologien.

Methoden: Systematische Literaturrecherche der Medizinbibliothek PubMed mit Inklusion von Reviews, die Bezug nehmen auf das Thema AR und VR mit Fokus auf Artikel, die das perioperative Feld berühren. Schlagwörter waren: Augmented Reality, Virtual Reality, Telementoring, Telechirurgie („Telesurgery“). Zudem Durchführung einer Recherche innerhalb der Grauliteratur zur Analyse der Investitionsbereitschaft im Gesundheitswesen durch private Technologiekonzerne.

Resultate Zum Thema „Augmented Reality“ wurden 1222 Artikel bei 119 Reviews und zu „Virtual Reality“ 7766 Artikel bei 878 Reviews angezeigt. Die Anzahl der veröffentlichten Artikel ist dabei über die letzten Jahre hinweg stark zunehmend. 45 Artikel wurden eingeschlossen. Multiple AR- und VR-Geräte sind bereits im operativen Setting etabliert. Die nächsten vielversprechenden Anwendungsbereiche werden hier v. a. die intraoperative Überlagerung von bildgebenden Verfahren über AR-Geräte sowie das Telementoring und die vermehrte Anwendung im Bereich operativer bzw. anatomischer Ausbildung betreffen. Die zu erhoffenden Vorteile wie Senkung der Komplikationsrate, Kosteneinsparungen, flächendeckender Wissenszuwachs und Optimierung des chirurgischen Ergebnisses müssen erst in wissenschaftlichen Studien nachgewiesen werden. Aus der Recherche der Grauliteratur geht ein enormes finanzielles Bestreben der Technologiekonzerne in diesem Sektor hervor.

Schlussfolgerung AR und VR werden aufgrund ihres enormen Verbesserungspotenzials zunehmend in den perioperativen Sektor integriert. Dabei ist der Nutzen dieser Technologien für relevante Endpunkte weitestgehend unklar. Diese sollten in gut-designten klinischen Studien rigoros getestet werden. Ärzte sollten diese technische Revolution aktiv mitgestalten, um das enorme Potenzial von AR und VR zum Wohle ihrer Patienten auszuschöpfen.

Abstract

Introduction Virtual reality (VR) is an artificially simulated environment permitting interaction. On the other hand, augmented reality (AR) is an enhanced version of reality created by the use of technology to overlay digital information on an image of something being viewed through a device. Both technologies have partially been implemented in clinical daily routine. Surgical applications of VR and AR are currently evaluated. Yet it is still unclear which possibilities these new and versatile applications offer for physicians.

Intention The goal of this article was to assess current and future use of AR und VR, with a special focus on surgery. We also summarised obstacles for AR and VR use as well as potential clinical improvements through these new technologies.

Methods Systematic literature research in PubMed with inclusion of reviews referring to AR and VR, especially focused on articles on surgery. Keywords were Augmented Reality, Virtual Reality, Telementoring and Telesurgery. Furthermore we briefly analysed the investment volume and investment strategy in medicine from the ten largest private technology companies of the USA.

Results The keyword “Augmented Reality” leads to 1222 articles and 119 reviews, while “Virtual Reality” offered 7766 articles and 878 reviews. In recent years, the amount of published articles has increased. 45 articles were included. Multiple AR- and VR-applications are already integrated in surgical daily routine. The next promising applications will concern the possibility of intraoperative overlap with radiological imaging via AR-tools, as well as telementoring and the use of AR and VR in surgical and anatomical education. The expected – but unproven – advantages include cost savings, reduction in complications, comprehensive knowledge acquisition and improvement in surgical results. In addition, we notice enormous financial investment by technology companies in this sector.

Conclusion Due to their tremendous potential, AR and VR technologies will be increasingly integrated into surgery. However, the benefit of these new technologies for relevant endpoints are currently unclear. This should be examined in rigorous clinical trials. Physicians should play a key role in this technological revolution to exploit the potential of AR and VR for their patients.

Schlüsselwörter

Augmented Reality - Virtual Reality - Telementoring - Telechirurgie

Keywords

Augmented Reality - Virtual Reality - Telementoring - Telesurgery

• Referenzen

• 1 Khor WS, Baker B, Amin K. et al. Augmented and virtual reality in surgery-the digital surgical environment: applications, limitations and legal pitfalls. Ann Transl Med 2016; 4 (23) 454
• 2 LaValle SM. 1 ed. Virtual Reality. Cambridge University Press; 2016
• 3 Hamacher A, Kim SJ, Cho ST. et al. Application of Virtual, Augmented, and Mixed Reality to Urology. Int Neurourol J 2016; 20 (03) 172-181
• 4 IE. S. The Ultimate Display. IFIP. Po, editor 1965.
• 5 Dan Curtis DM, Peter Gruenbaum et al. Several devils in the details: making an AR application work in the airplane factory. IWAR ʼ98 Proceedings of the international workshop on Augmented reality: placing artificial objects in real scenes: placing artificial objects in real scenes 1999.
• 6 Singer N. How Big Tech Is Going After Your Health Care. The NewYork Times; 26.12.2017
• 7 CB Insights-Where Big Tech Is Placing Bets In Healthcare [Internet]. December 28, 2018.
• 8 Shah A. Intel tunes its mega-chip for machine learning. PC World. 2016.
• 9 Medical Realities. Berwick Street, W1F 8RG. London, UK: Copyright 2017 Medical Realities Ltd: 26 | [phone]. https://www.medicalrealities.com/
• 10 Reidsma D, Haruhiro K, Nijholt A. et al. Advances in Computer Entertainment Technology. 10th International Conference. Boekelo, The Netherlands: ACE 2013; 2013
• 11 TechSpecs. Google 2018 https://support.google.com/glass/answer/3064128?hl=en-GB [Stand: 20.09.2018]
• 12 Muensterer OJ, Lacher M, Zoeller C. et al. Google Glass in pediatric surgery: an exploratory study. Int J Surg 2014; 12: 281-289
• 13 HoloLens Hardware details. Im Internet: https://docs.microsoft.com/en-us/windows/mixed-reality/hololens-hardware-details Stand 05.11.2018
• 14 Shah J, Mackay S, Vale J. et al. Simulation in urology -- a role for virtual reality?. BJU Int 2001; 88: 661-665
• 15 Ma M, Fallavollita P, Seelbach I. et al. Personalized augmented reality for anatomy education. Clin Anat 2016; 29: 446-453
• 16 Estai M, Bunt S. Best teaching practices in anatomy education: A critical review. Ann Anat 2016; 208: 151-157
• 17 Benninger B. Google Glass, ultrasound and palpation: the anatomy teacher of the future?. Clin Anat 2015; 28: 152-155
• 18 Kamphuis C, Barsom E, Schijven M. et al. Augmented reality in medical education?. Perspect Med Educ 2014; 3: 300-311
• 19 Moro C, Stromberga Z, Raikos A. et al. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat Sci Educ 2017; 10: 549-559
• 20 Wilson MS, Middlebrook A, Sutton C. et al. MIST VR: a virtual reality trainer for laparoscopic surgery assesses performance. Ann R Coll Surg Engl 1997; 79: 403-404
• 21 Al-Qattan MM, Al-Turaiki TM. Flexor tendon repair in zone 2 using a six-strand „figure of eigh“ suture. J Hand Surg Eur Vol 2009; 34: 322-328
• 22 Angeles JG, Heminger H, Mass DP. Comparative biomechanical performances of 4-strand core suture repairs for zone II flexor tendon lacerations. J Hand Surg Am 2002; 27: 508-517
• 23 Burdea G, Patounakis G, Popescu V. et al. Virtual reality-based training for the diagnosis of prostate cancer. IEEE Trans Biomed Eng 1999; 46: 1253-1260
• 24 Fiard G, Selmi SY, Promayon E. et al. Initial validation of a virtual-reality learning environment for prostate biopsies: realism matters!. J Endourol 2014; 28: 453-458
• 25 Khan R, Aydin A, Khan MS. et al. Simulation-based training for prostate surgery. BJU Int 2015; 116: 665-674
• 26 Tang JB, Amadio PC, Boyer MI. et al. Current practice of primary flexor tendon repair: a global view. Hand Clin 2013; 29: 179-189
• 27 Armstrong DG, Rankin TM, Giovinco NA. et al. A heads-up display for diabetic limb salvage surgery: a view through the google looking glass. J Diabetes Sci Technol 2014; 8: 951-956
• 28 Gardiner S, Hartzell TL. Telemedicine and plastic surgery: a review of its applications, limitations and legal pitfalls. J Plast Reconstr Aesthet Surg 2012; 65: e47-53
• 29 Glenn IC, Bruns NE, Hayek D. et al. Rural surgeons would embrace surgical telementoring for help with difficult cases and acquisition of new skills. Surg Endosc 2017; 31: 1264-1268
• 30 Miyake RK, Zeman HD, Duarte FH. et al. Vein imaging: a new method of near infrared imaging, where a processed image is projected onto the skin for the enhancement of vein treatment. Dermatol Surg 2006; 32: 1031-1038
• 31 Brouillette D, Thivierge G, Marchand D. et al. Preparative study regarding the implementation of a muscular fatigue model in a virtual task simulator. Work 2012; 41: 2216-2225
• 32 Glockner JF. Navigating the aorta: MR virtual vascular endoscopy. Radiographics 2003; 23: e11
• 33 Lovo EE, Quintana JC, Puebla MC. et al. A novel, inexpensive method of image coregistration for applications in image-guided surgery using augmented reality. Neurosurgery 2007; 60: 366-371 discussion 71–72
• 34 Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery 2014; 10: 252-260 discussion 60–61
• 35 Alberti O, Dorward NL, Kitchen ND. et al. Neuronavigation--impact on operating time. Stereotact Funct Neurosurg 1997; 68: 44-48
• 36 R. Azuma YB, R. Behringer et al. Recent advances in augmented reality. IEEE Computer Graphics and Applications (Volume: 21, Issue: 6): https://ieeexplore.ieee.org/document/963459/ (Stand:20.09.2018); 2001.
• 37 Volonte F, Pugin F, Bucher P. et al. Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 2011; 18: 506-509
• 38 Su LM, Vagvolgyi BP, Agarwal R. et al. Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 2009; 73: 896-900
• 39 Schnelzer A, Ehlerding A, Blumel C. et al. Showcase of Intraoperative 3D Imaging of the Sentinel Lymph Node in a Breast Cancer Patient using the New Freehand SPECT Technology. Breast Care (Basel) 2012; 7: 484-486
• 40 Heuveling DA, Karagozoglu KH, van Schie A. et al. Sentinel node biopsy using 3D lymphatic mapping by freehand SPECT in early stage oral cancer: a new technique. Clin Otolaryngol 2012; 37: 89-90
• 41 Tagaya N, Yamazaki R, Nakagawa A. et al. Intraoperative identification of sentinel lymph nodes by near-infrared fluorescence imaging in patients with breast cancer. Am J Surg 2008; 195: 850-853
• 42 Wake N, Bjurlin MA, Rostami P. et al. Three-dimensional Printing and Augmented Reality: Enhanced Precision for Robotic Assisted Partial Nephrectomy. Urology 2018; 116: 227-228
• 43 Porpiglia F, Bertolo R, Amparore D. et al. Augmented reality during robot-assisted radical prostatectomy: expert robotic surgeons' on-the-spot insights after live surgery. Minerva Urol Nefrol 2018; 70: 226-229
• 44 Yoon JW, Chen RE, Kim EJ. et al. Augmented reality for the surgeon: Systematic review. Int J Med Robot 2018; 14: e1914
• 45 Borgmann H, Rodriguez Socarras M, Salem J. et al. Feasibility and safety of augmented reality-assisted urological surgery using smartglass. World J Urol 2017; 35: 967-972
• 46 Hughes-Hallett A, Mayer EK, Marcus HJ. et al. Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 2014; 83: 266-273
• 47 Davis MC, Can DD, Pindrik J. et al. Virtual Interactive Presence in Global Surgical Education: International Collaboration Through Augmented Reality. World Neurosurg 2016; 86: 103-111
• 48 Shenai MB, Dillavou M, Shum C. et al. Virtual interactive presence and augmented reality (VIPAR) for remote surgical assistance. Neurosurgery 2011; 68: 200-207 discussion 7
• 49 Wu YF, Tang JB. Recent developments in flexor tendon repair techniques and factors influencing strength of the tendon repair. J Hand Surg Eur Vol 2014; 39: 6-19
• 50 Kumcu A, Vermeulen L, Elprama SA. et al. Effect of video lag on laparoscopic surgery: correlation between performance and usability at low latencies. Int J Med Robot 2017;
• 51 Marescaux J, Leroy J, Rubino F. et al. Transcontinental robot-assisted remote telesurgery: feasibility and potential applications. Ann Surg 2002; 235: 487-492
• 52 Tzou CH, Artner NM, Pona I. et al. Comparison of three-dimensional surface-imaging systems. J Plast Reconstr Aesthet Surg 2014; 67: 489-497
• 53 Hoffman HG, Meyer 3rd WJ. et al. Feasibility of articulated arm mounted Oculus Rift Virtual Reality goggles for adjunctive pain control during occupational therapy in pediatric burn patients. Cyberpsychol Behav Soc Netw 2014; 17: 397-401
• 54 Cao Y, Tang JB. Biomechanical evaluation of a four-strand modification of the Tang method of tendon repair. J Hand Surg Br 2005; 30: 374-378