Voice interface for semi-automatic generation of annotated surgical reports of paranasal sinus surgeries

Background The aim of our project is the development of an annotation tool that uses speech recognition during a paranasal sinus operation to create a surgical report. This is intended to save time in documentation and improve its quality. We consider this speech interface to be the base technology for the development of future collaborative assistance functions.

Methods In a first step an annotation tool for surgical steps from recorded paranasal sinus operations was developed. This tool uses a trained language model, which converts speech inputs into a report document. This document is completed by video frames of the recording, which are recognized by defined keywords and are relevant for the surgical procedure. Using the developed tool, three video recordings of paranasal sinus operations were voice annotated and thus surgical reports were generated. Using an ordinal scaled questionnaire, both the retrospective and the voice annotated surgery reports were evaluated.

Results The average rating of the retrospectively written surgical reports was "2.5", the mean rating of the voice annotated ones was "1.61". In principle, the speech annotation is expected to provide benefits in the level of detail of the generated reports. The daily time saving is estimated at 0-30 minutes. In addition, the natural speech annotations provide a detailed description of the recorded video material

Conclusion The use of a speech interface for annotation tasks offers the potential of high-quality and time-saving surgical documentation. Furthermore, we believe that speech recognition of surgical procedures provides a high potential for the development of new integrated assistance systems.

Poster-PDF A-1559.pdf

Teile der Arbeit entstanden mithilfe der Förderung des Bundesministeriums für Bildung und Forschung im Projekt „COMPAS“ unter dem Förderkennzeichen 16SV8017.

Conflict of interest

Der Erstautor gibt keinen Interessenskonflikt an.

Publication History

Article published online:
13 May 2021