Download PDF
Methods Inf Med 2022; 61(03/04): 084-089
DOI: 10.1055/s-0042-1749358
Original Article

Automated Identification of Clinical Procedures in Free-Text Electronic Clinical Records with a Low-Code Named Entity Recognition Workflow

Carmelo Macri
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
3   Discipline of Ophthalmology and Visual Sciences, The University of Adelaide, Adelaide, South Australia, Australia
,
Ian Teoh
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
,
Stephen Bacchi
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
Michelle Sun
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
Dinesh Selva
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
Robert Casson
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
,
WengOnn Chan
1   Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia
2   Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia
› Author Affiliations Funding None.
› Further Information
    Permissions and Reprints

Abstract

Background Clinical procedures are often performed in outpatient clinics without prior scheduling at the administrative level, and documentation of the procedure often occurs solely in free-text clinical electronic notes. Natural language processing (NLP), particularly named entity recognition (NER), may provide a solution to extracting procedure data from free-text electronic notes.

Methods Free-text notes from outpatient ophthalmology visits were collected from the electronic clinical records at a single institution over 3 months. The Prodigy low-code annotation tool was used to create an annotation dataset and train a custom NER model for clinical procedures. Clinical procedures were extracted from the entire set of clinical notes.

Results There were a total of 5,098 clinic notes extracted for the study period; 1,923 clinic notes were used to build the NER model, which included a total of 231 manual annotations. The NER model achieved an F-score of 0.767, a precision of 0.810, and a recall of 0.729. The most common procedures performed included intravitreal injections of therapeutic substances, removal of corneal foreign bodies, and epithelial debridement of corneal ulcers.

Conclusion The use of a low-code annotation software tool allows the rapid creation of a custom annotation dataset to train a NER model to identify clinical procedures stored in free-text electronic clinical notes. This enables clinicians to rapidly gather previously unidentified procedural data for quality improvement and auditing purposes. Low-code annotation tools may reduce time and coding barriers to clinician participation in NLP research.

Keywords

annotation tool - named entity recognition - natural language processing - electronic health records - low-code - workflow


Publication History

Received: 02 December 2021

Accepted: 13 April 2022

Article published online:
12 September 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany