Download PDF
CC BY-NC-ND 4.0 · Endosc Int Open 2024; 12(07): E849-E853
DOI: 10.1055/a-2333-8138
Original article

Assessment of colonoscopy skill using machine learning to measure quality: Proof-of-concept and initial validation

Matthew Wittbrodt
1   Information Services, Northwestern Medicine, Chicago, United States
,
Matthew Klug
1   Information Services, Northwestern Medicine, Chicago, United States
,
Mozziyar Etemadi
1   Information Services, Northwestern Medicine, Chicago, United States
2   Anesthesiology, Northwestern University Feinberg School of Medicine, Chicago, United States (Ringgold ID: RIN12244)
,
Anthony Yang
3   Surgery, Indiana University School of Medicine, Indianapolis, United States (Ringgold ID: RIN12250)
,
John E. Pandolfino
4   Medicine, Northwestern University Feinberg School of Medicine, Chicago, United States (Ringgold ID: RIN12244)
,
Rajesh N. Keswani
4   Medicine, Northwestern University Feinberg School of Medicine, Chicago, United States (Ringgold ID: RIN12244)
› Author AffiliationsSupported by: Betty and Gordon Moore Foundation
Supported by: Digestive Health Foundation
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background and study aims Low-quality colonoscopy increases cancer risk but measuring quality remains challenging. We developed an automated, interactive assessment of colonoscopy quality (AI-CQ) using machine learning (ML).

Methods Based on quality guidelines, metrics selected for AI development included insertion time (IT), withdrawal time (WT), polyp detection rate (PDR), and polyps per colonoscopy (PPC). Two novel metrics were also developed: HQ-WT (time during withdrawal with clear image) and WT-PT (withdrawal time subtracting polypectomy time). The model was pre-trained using a self-supervised vision transformer on unlabeled colonoscopy images and then finetuned for multi-label classification on another mutually exclusive colonoscopy image dataset. A timeline of video predictions and metric calculations were presented to clinicians in addition to the raw video using a web-based application. The model was externally validated using 50 colonoscopies at a second hospital.

Results The AI-CQ accuracy to identify cecal intubation was 88%. IT (P = 0.99) and WT (P = 0.99) were highly correlated between manual and AI-CQ measurements with a median difference of 1.5 seconds and 4.5 seconds, respectively. AI-CQ PDR did not significantly differ from manual PDR (47.6% versus 45.5%, P = 0.66). Retroflexion was correctly identified in 95.2% and number of right colon evaluations in 100% of colonoscopies. HQ-WT was 45.9% of, and significantly correlated with (P = 0.85) WT time.

Conclusions An interactive AI assessment of colonoscopy skill can automatically assess quality. We propose that this tool can be utilized to rapidly identify and train providers in need of remediation.

Keywords

Quality and logistical aspects - Training - Quality management


Publication History

Received: 22 April 2024

Accepted after revision: 14 May 2024

Accepted Manuscript online:
27 May 2024

Article published online:
03 July 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial-License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Do A, Weinberg J, Kakkar A. et al. Reliability of adenoma detection rate is based on procedural volume. Gastrointest Endosc 2013; 77: 376-380
    CrossrefPubMedSearch in Google Scholar
  • 2 Pace D, Borgaonkar M, Evans B. et al. Annual colonoscopy volume and maintenance of competency for surgeons. Surg Endosc 2017; 31: 2630-2635
    CrossrefPubMedSearch in Google Scholar
  • 3 Duloy A, Yadlapati RH, Benson M. et al. Video-based assessments of colonoscopy inspection quality correlate with quality metrics and highlight areas for improvement. Clin Gastroenterol Hepatol 2019; 17: 691-700
    CrossrefPubMedSearch in Google Scholar
  • 4 Duloy AM, Kaltenbach TR, Wood M. et al. Colon polypectomy report card improves polypectomy competency: results of a prospective quality improvement study (with video). Gastrointest Endosc 2019; 89: 1212-1221
    CrossrefPubMedSearch in Google Scholar
  • 5 Keswani RN, Byrd D, Garcia Vicente F. et al. Amalgamation of cloud-based colonoscopy videos with patient-level metadata to facilitate large-scale machine learning. Endosc Int Open 2021; 9: E233-E238
    Thieme ConnectPubMedSearch in Google Scholar
  • 6 Shaukat A, Lichtenstein DR, Somers SC. et al. Computer-aided detection improves adenomas per colonoscopy for screening and surveillance colonoscopy: A randomized trial. Gastroenterology 2022; 163: 732-741
    CrossrefPubMedSearch in Google Scholar
  • 7 Repici A, Badalamenti M, Maselli R. et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520 e517
    CrossrefPubMedSearch in Google Scholar
  • 8 Glissen Brown JR, Mansour NM, Wang P. et al. Deep learning computer-aided polyp detection reduces adenoma miss rate: A United States multi-center randomized tandem colonoscopy study (CADeT-CS Trial). Clin Gastroenterol Hepatol 2022; 20: 1499-1507 e1494
    CrossrefPubMedSearch in Google Scholar
  • 9 Thakkar S, Carleton NM, Rao B. et al. Use of artificial intelligence-based analytics from live colonoscopies to optimize the quality of the colonoscopy examination in real time: proof of concept. Gastroenterology 2020; 158: 1219-1221 e1212
    CrossrefPubMedSearch in Google Scholar
  • 10 Barua I, Misawa M, Glissen Brown JR. et al. Speedometer for withdrawal time monitoring during colonoscopy: a clinical implementation trial. Scand J Gastroenterol 2023; 58: 664-670
    CrossrefPubMedSearch in Google Scholar
  • 11 Lux TJ, Sassmanshausen Z, Herold K. et al. Assisted documentation as new focus for artificial intelligence in endoscopy: The precedent of reliable withdrawal time and image reporting. Endoscopy 2023; 55: 1118-1123
    Thieme ConnectPubMedSearch in Google Scholar
  • 12 Kaltenbach T, Patel SG, Nguyen-Vu T. et al. Varied trainee competence in cold snare polypectomy – results of the COMPLETE randomized controlled trial. Am J Gastroenterol 2023; 118: 1880-1887
    CrossrefPubMedSearch in Google Scholar