Download PDF
CC BY-NC-ND 4.0 · Endoscopy
DOI: 10.1055/a-2296-5696
Original article

Influence of artificial intelligence on the diagnostic performance of endoscopists in the assessment of Barrett’s esophagus: a tandem randomized and video trial

Michael Meinikheim  ‡
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Robert Mendel‡
2   Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
,
Christoph Palm
2   Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
,
Andreas Probst
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Anna Muzalyova
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Markus W. Scheppach
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Sandra Nagl
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Elisabeth Schnoy
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Christoph Römmele 
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Dominik A. H. Schulz
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Jakob Schlottmann
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Friederike Prinz
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
David Rauber
2   Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
,
Tobias Rückert
2   Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
,
Tomoaki Matsumura 
3   Department of Gastroenterology, Chiba University Graduate School of Medicine, Chiba, Japan
,
Glòria Fernández-Esparrach
4   Endoscopy Unit, Gastroenterology Department, ICMDM, Hospital Clínic de Barcelona, Barcelona, Spain
5   Faculty of Medicine, University of Barcelona, Barcelona, Spain
6   Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
7   Centro de Investigación Biomédica en Red de Enfermedades Hepáticas y Digestivas (CIBEREHD), Barcelona, Spain
,
Nasim Parsa
8   Division of Gastroenterology and Hepatology, Mayo Clinic, Scottsdale, United States
9   Satisfai Health, Vancouver, Canada
,
Michael F. Byrne
9   Satisfai Health, Vancouver, Canada
10   Gastroenterology, Vancouver General Hospital, The University of British Columbia, Vancouver, Canada
,
Helmut Messmann
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
,
Alanna Ebigbo
1   Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany (Ringgold ID: RIN39694)
› Author Affiliations
› Further Information
Also available at
   Permissions and Reprints


Referred to by:

Artificial intelligence (AI) systems for detection of Barrett’s neoplasia: time to bridge domain gaps and explore human–AI interactionEndoscopy eFirst
DOI: 10.1055/a-2335-1331

Abstract

Background This study evaluated the effect of an artificial intelligence (AI)-based clinical decision support system on the performance and diagnostic confidence of endoscopists in their assessment of Barrett’s esophagus (BE).

Methods 96 standardized endoscopy videos were assessed by 22 endoscopists with varying degrees of BE experience from 12 centers. Assessment was randomized into two video sets: group A (review first without AI and second with AI) and group B (review first with AI and second without AI). Endoscopists were required to evaluate each video for the presence of Barrett’s esophagus-related neoplasia (BERN) and then decide on a spot for a targeted biopsy. After the second assessment, they were allowed to change their clinical decision and confidence level.

Results AI had a stand-alone sensitivity, specificity, and accuracy of 92.2%, 68.9%, and 81.3%, respectively. Without AI, BE experts had an overall sensitivity, specificity, and accuracy of 83.3%, 58.1%, and 71.5%, respectively. With AI, BE nonexperts showed a significant improvement in sensitivity and specificity when videos were assessed a second time with AI (sensitivity 69.8% [95%CI 65.2%–74.2%] to 78.0% [95%CI 74.0%–82.0%]; specificity 67.3% [95%CI 62.5%–72.2%] to 72.7% [95%CI 68.2%–77.3%]). In addition, the diagnostic confidence of BE nonexperts improved significantly with AI.

Conclusion BE nonexperts benefitted significantly from additional AI. BE experts and nonexperts remained significantly below the stand-alone performance of AI, suggesting that there may be other factors influencing endoscopists’ decisions to follow or discard AI advice.

Supplementary Material



Publication History

Received: 19 September 2023

Accepted after revision: 13 March 2024

Accepted Manuscript online:
28 March 2024

Article published online:
02 May 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial-License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Hvid-Jensen F, Pedersen L, Drewes AM. et al. Incidence of adenocarcinoma among patients with Barrett’s esophagus. N Engl J Med 2011; 365: 1375-1383
    CrossrefPubMedGoogle Scholar
  • 2 Coleman HG, Xie SH, Lagergren J. The epidemiology of esophageal adenocarcinoma. Gastroenterology 2018; 154: 390-405 DOI: 10.1053/j.gastro.2017.07.046. (PMID: 28780073)
    CrossrefPubMedGoogle Scholar
  • 3 Sung H, Ferlay J, Siegel RL. et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2021; 71: 209-249 DOI: 10.3322/caac.21660. (PMID: 33538338)
    CrossrefPubMedGoogle Scholar
  • 4 Smyth EC, Lagergren J, Fitzgerald RC. et al. Oesophageal cancer. Nat Rev Dis Primers 2017; 3: 17048 DOI: 10.1038/nrdp.2017.48. (PMID: 28748917)
    CrossrefPubMedGoogle Scholar
  • 5 Visrodia K, Singh S, Krishnamoorthi R. et al. Magnitude of missed esophageal adenocarcinoma after Barrett’s esophagus diagnosis: a systematic review and meta-analysis. Gastroenterology 2016; 150: 599-607.e597
    CrossrefPubMedGoogle Scholar
  • 6 Messmann H, Ebigbo A, Hassan C. et al. How to integrate artificial intelligence in gastrointestinal practice. Gastroenterology 2022; 162: 1583-1586 DOI: 10.1053/j.gastro.2022.02.029. (PMID: 35196540)
    CrossrefPubMedGoogle Scholar
  • 7 van der Sommen F, Zinger S, Curvers WL. et al. Computer-aided detection of early neoplastic lesions in Barrett’s esophagus. Endoscopy 2016; 48: 617-624 DOI: 10.1055/s-0042-105284. (PMID: 27100718)
    Article in Thieme ConnectPubMedGoogle Scholar
  • 8 de Groof AJ, Struyvenberg MR, van der Putten J. et al. Deep-learning system detects neoplasia in patients with Barrett’s esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking. Gastroenterology 2020; 158: 915-929.e914
    CrossrefPubMedGoogle Scholar
  • 9 de Groof AJ, Struyvenberg MR, Fockens KN. et al. Deep learning algorithm detection of Barrett’s neoplasia with high accuracy during live endoscopic procedures: a pilot study (with video). Gastrointest Endosc 2020; 91: 1242-1250
    CrossrefPubMedGoogle Scholar
  • 10 Hashimoto R, Requa J, Dao T. et al. Artificial intelligence using convolutional neural networks for real-time detection of early esophageal neoplasia in Barrett’s esophagus (with video). Gastrointest Endosc 2020; 91: 1264-1271.e1261
    CrossrefPubMedGoogle Scholar
  • 11 Iwagami H, Ishihara R, Aoyama K. et al. Artificial intelligence for the detection of esophageal and esophagogastric junctional adenocarcinoma. J Gastroenterol Hepatol 2021; 36: 131-136 DOI: 10.1111/jgh.15136. (PMID: 32511793)
    CrossrefPubMedGoogle Scholar
  • 12 Struyvenberg MR, de Groof AJ, van der Putten J. et al. A computer-assisted algorithm for narrow-band imaging-based tissue characterization in Barrett’s esophagus. Gastrointest Endosc 2021; 93: 89-98 DOI: 10.1016/j.gie.2020.05.050. (PMID: 32504696)
    CrossrefPubMedGoogle Scholar
  • 13 Hussein M, González-Bueno Puyal J, Lines D. et al. A new artificial intelligence system successfully detects and localises early neoplasia in Barrett’s esophagus by using convolutional neural networks. United European Gastroenterol J 2022; 10: 528-537
    CrossrefPubMedGoogle Scholar
  • 14 Ebigbo A, Mendel R, Probst A. et al. Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma. Gut 2019; 68: 1143-1145 DOI: 10.1136/gutjnl-2018-317573. (PMID: 30510110)
    CrossrefPubMedGoogle Scholar
  • 15 Messmann H, Bisschops R, Antonelli G. et al. Expected value of artificial intelligence in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2022; 54: 1211-1231 DOI: 10.1055/a-1950-5694. (PMID: 36270318)
    Article in Thieme ConnectPubMedGoogle Scholar
  • 16 Vasey B, Nagendran M, Campbell B. et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med 2022; 28: 924-933 DOI: 10.1038/s41591-022-01772-9. (PMID: 35585198)
    CrossrefPubMedGoogle Scholar
  • 17 Chen L-C, Zhu Y, Papandreou G. et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. et al. In: Ferrari V, Hebert M, Sminchisescu C. Computer Vision – ECCV 2018. Cham: Springer International Publishing; 2018: 833-851
    Google Scholar
  • 18 Huang Y, Wang Q, Jia W. et al. See more than once: kernel-sharing atrous convolution for semantic segmentation. Neurocomputing 2021; 443: 26-34
    CrossrefPubMedGoogle Scholar
  • 19 He K, Zhang X, Ren S. et al. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016; 770-778
    CrossrefPubMedGoogle Scholar
  • 20 Mendel R, Rauber D, de Souza Jr LA. et al. Error-Correcting Mean-Teacher: corrections instead of consistency-targets applied to semi-supervised medical image segmentation. Comput Biol Med 2023; 154: 106585 DOI: 10.1016/j.compbiomed.2023.106585. (PMID: 36731360)
    CrossrefPubMedGoogle Scholar
  • 21 Singer ME, Odze RD. High rate of missed Barrett’s esophagus when screening with forceps biopsies. Esophagus 2023; 20: 143-149 DOI: 10.1007/s10388-022-00943-4. (PMID: 35864425)
    CrossrefPubMedGoogle Scholar
  • 22 Ebigbo A, Mendel R, Probst A. et al. Real-time use of artificial intelligence in the evaluation of cancer in Barrett’s oesophagus. Gut 2020; 69: 615-616 DOI: 10.1136/gutjnl-2019-319460. (PMID: 31541004)
    CrossrefPubMedGoogle Scholar
  • 23 Meinikheim M, Mendel R, Scheppach MW. et al. Influence of an artificial intelligence (AI) based decision support system (DSS) on the diagnostic performance of non-experts in Barrett’s esophagus related neoplasia (BERN). Endoscopy 2022; 54: OP076
    PubMedGoogle Scholar
  • 24 Weusten B, Bisschops R, Coron E. et al. Endoscopic management of Barrett’s esophagus: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2017; 49: 191-198 DOI: 10.1055/s-0042-122140. (PMID: 28122386)
    Article in Thieme ConnectPubMedGoogle Scholar
  • 25 Wu L, Shang R, Sharma P. et al. Effect of a deep learning-based system on the miss rate of gastric neoplasms during upper gastrointestinal endoscopy: a single-centre, tandem, randomised controlled trial. Lancet Gastroenterol Hepatol 2021; 6: 700-708
    CrossrefPubMedGoogle Scholar
  • 26 Glissen Brown JR, Mansour NM, Wang P. et al. Deep learning computer-aided polyp detection reduces adenoma miss rate: a United States multi-center randomized tandem colonoscopy study (CADeT-CS Trial). Clin Gastroenterol Hepatol 2022; 20: 1499-1507.e1494
    CrossrefPubMedGoogle Scholar
  • 27 Wallace MB, Sharma P, Bhandari P. et al. Impact of artificial intelligence on miss rate of colorectal neoplasia. Gastroenterology 2022; 163: 295-304.e295
    CrossrefPubMedGoogle Scholar
  • 28 Fockens KN, Jukema JB, Boers T. et al. Towards a robust and compact deep learning system for primary detection of early Barrett’s neoplasia: initial image-based results of training on a multi-center retrospectively collected data set. United European Gastroenterol J 2023; 11: 324-336 DOI: 10.1002/ueg2.12363. (PMID: 37095718)
    CrossrefPubMedGoogle Scholar
  • 29 Abdelrahim M, Saiko M, Maeda N. et al. Development and validation of artificial neural networks model for detection of Barrett’s neoplasia: a multicenter pragmatic nonrandomized trial (with video). Gastrointest Endosc 2023; 97: 422-434
    CrossrefPubMedGoogle Scholar