Validation of an Automated Symptom-Based Triage Tool in Ophthalmology

Abstract

Objectives Acute care ophthalmic clinics often suffer from inefficient triage, leading to suboptimal patient access and resource utilization. This study reports the preliminary results of a novel, symptom-based, patient-directed, online triage tool developed to address the most common acute ophthalmic diagnoses and associated presenting symptoms.

Methods A retrospective chart review of patients who presented to a tertiary academic medical center's urgent eye clinic after being referred for an urgent, semi-urgent, or nonurgent visit by the ophthalmic triage tool between January 1, 2021 and January 1, 2022 was performed. Concordance between triage category and severity of diagnosis on the subsequent clinic visit was assessed.

Results The online triage tool was utilized 1,370 and 95 times, by the call center administrators (phone triage group) and patients directly (web triage group), respectively. Of all patients triaged with the tool, 8.50% were deemed urgent, 59.2% semi-urgent, and 32.3% nonurgent. At the subsequent clinic visit, the history of present illness had significant agreement with symptoms reported to the triage tool (99.3% agreement, weighted kappa = 0.980, p < 0.001). The triage algorithm also had significant agreement with the severity of the physician diagnosis (97.0% agreement, weighted kappa = 0.912, p < 0.001). Zero patients were found to have a diagnosis on exam that should have corresponded to a higher urgency level on the triage tool.

Conclusion The automated ophthalmic triage algorithm was able to safely and effectively triage patients based on symptoms. Future work should focus on the utility of this tool to reduce nonurgent patient load in urgent clinical settings and to improve access for patients who require urgent medical care.

Keywords

urgent care - ophthalmic emergency - triage - automated triage tool - symptom-based triage tool

Note

This study was presented at the Association for Research in Vision and Ophthalmology Meeting 2022.

Ethics Statement

This study was deemed exempt by the University of Pennsylvania Institutional Review Board as it was considered a quality improvement study with no risk or minimal risk to subjects, with all secondary analyses performed on nonidentifiable data.

Authors' Contributions

All authors contributed to the planning, conducting, and reporting of the work described in the article. M.S.R. and E.M. as guarantors for content accept full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish. E.M. attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. All listed authors have approved the present version of the manuscript.

Protection of Human and Animal Subjects

No interventions were performed on human subjects.

^# Co-first authors. Elana Meer and Meera S. Ramakrishnan contributed equally to this work.

Publication History

Received: 30 November 2022

Accepted: 27 March 2023

Accepted Manuscript online:
29 March 2023

Article published online:
07 June 2023

• References

• 1 CDC. Social distancing, quarantine, and isolation. CDC.gov, 2020. Accessed April 14, 2023 at: https://www.cdc.gov/quarantine/index.html
• 2 Saleem SM, Pasquale LR, Sidoti PA, Tsai JC. Virtual ophthalmology: telemedicine in a COVID-19 era. Am J Ophthalmol 2020; 216: 237-242
• 3 Channa R, Zafar SN, Canner JK, Haring RS, Schneider EB, Friedman DS. Epidemiology of eye-related emergency department visits. JAMA Ophthalmol 2016; 134 (03) 312-319
• 4 Nari J, Allen LH, Bursztyn LLCD. Accuracy of referral diagnosis to an emergency eye clinic. Can J Ophthalmol 2017; 52 (03) 283-286
• 5 Docherty G, Hwang J, Yang M. et al. Prospective analysis of emergency ophthalmic referrals in a Canadian tertiary teaching hospital. Can J Ophthalmol 2018; 53 (05) 497-502
• 6 Deaner JD, Amarasekera DC, Ozzello DJ. et al. Accuracy of referral and phone-triage diagnoses in an eye emergency department. Ophthalmology 2021; 128 (03) 471-473
• 7 Shen BY, Salman AR, Shah SM. et al. Clinical outcomes following implementation of a formalized “flashes and floaters” emergency department triage protocol. Am J Ophthalmol 2022; 242 (00) 125-130
• 8 Shen C, Nguyen M, Gregor A, Isaza G, Beattie A. Accuracy of a popular online symptom checker for ophthalmic diagnoses. JAMA Ophthalmol 2019; 137 (06) 690-692
• 9 Bourdon H, Jaillant R, Ballino A. et al. Teleconsultation in primary ophthalmic emergencies during the COVID-19 lockdown in Paris: experience with 500 patients in March and April 2020. J Fr Ophtalmol 2020; 43 (07) 577-585
• 10 Chen Y, Ismail R, Cheema MR, Ting DSJ, Masri I. Implementation of a new telephone triage system in ophthalmology emergency department during COVID-19 pandemic: clinical effectiveness, safety and patient satisfaction. Eye (Lond) 2022; 36 (05) 1126-1128
• 11 Eijk ESV, Bettink-Remeijer MW, Timman R, Busschbach JJV. From pen-and-paper questionnaire to a computer-assisted instrument for self-triage in the ophthalmic emergency department: process and validation. Comput Biol Med 2015; 66: 258-262
• 12 Eijk ESV, Wefers Bettink-Remeijer M, Timman R, Heres MHB, Busschbach JJV. Criterion validity of a computer-assisted instrument of self-triage (ca-ISET) compared to the validity of regular triage in an ophthalmic emergency department. Int J Med Inform 2016; 85 (01) 61-67
• 13 Rossi T, Boccassini B, Iossa M, Mutolo MG, Lesnoni G, Mutolo PA. Triaging and coding ophthalmic emergency: the Rome Eye Scoring System for Urgency and Emergency (RESCUE): a pilot study of 1,000 eye-dedicated emergency room patients. Eur J Ophthalmol 2007; 17 (03) 413-417
• 14 D'Oria F, Bordinone MA, Rizzo T. et al. Validation of a new system for triage of ophthalmic emergencies: the Alphabetical Triage Score for Ophthalmology (ATSO). Int Ophthalmol 2020; 40 (09) 2291-2296
• 15 Meer EA, Herriman M, Lam D. et al. Design, implementation, and validation of an automated, algorithmic COVID-19 triage tool. Appl Clin Inform 2021; 12 (05) 1021-1028
• 16 Gerstenblith AT, Rabinowitz MP. The Wills Eye Manual: Office and Emergency Room Diagnosis and Treatment of Eye Disease. Philadelphia, PA: Lippincott Williams & Wilkins; 1990
• 17 Kubba H. Reading skills of otolaryngology outpatients: implications for information provision. J Laryngol Otol 2000; 114 (09) 694-696
• 18 Penn Medicine. Penn Medicine Direct. Accessed July 19, 2022 at: https://direct.pennmedicine.org/vision
• 19 Scanzera AC, Chang AY, Valikodath N. et al. Assessment of a novel ophthalmology tele-triage system during the COVID-19 pandemic. BMC Ophthalmol 2021; 21 (01) 346