Influence of artificial intelligence on the diagnostic performance of endoscopists in the assessment of Barrett’s esophagus: a tandem randomized and video trial

Abstract

Background This study evaluated the effect of an artificial intelligence (AI)-based clinical decision support system on the performance and diagnostic confidence of endoscopists in their assessment of Barrett’s esophagus (BE).

Methods 96 standardized endoscopy videos were assessed by 22 endoscopists with varying degrees of BE experience from 12 centers. Assessment was randomized into two video sets: group A (review first without AI and second with AI) and group B (review first with AI and second without AI). Endoscopists were required to evaluate each video for the presence of Barrett’s esophagus-related neoplasia (BERN) and then decide on a spot for a targeted biopsy. After the second assessment, they were allowed to change their clinical decision and confidence level.

Results AI had a stand-alone sensitivity, specificity, and accuracy of 92.2%, 68.9%, and 81.3%, respectively. Without AI, BE experts had an overall sensitivity, specificity, and accuracy of 83.3%, 58.1%, and 71.5%, respectively. With AI, BE nonexperts showed a significant improvement in sensitivity and specificity when videos were assessed a second time with AI (sensitivity 69.8% [95%CI 65.2%–74.2%] to 78.0% [95%CI 74.0%–82.0%]; specificity 67.3% [95%CI 62.5%–72.2%] to 72.7% [95%CI 68.2%–77.3%]). In addition, the diagnostic confidence of BE nonexperts improved significantly with AI.

Conclusion BE nonexperts benefitted significantly from additional AI. BE experts and nonexperts remained significantly below the stand-alone performance of AI, suggesting that there may be other factors influencing endoscopists’ decisions to follow or discard AI advice.

Publikationsverlauf

Eingereicht: 19. September 2023

Angenommen nach Revision: 13. März 2024

Accepted Manuscript online:
28. März 2024

Artikel online veröffentlicht:
02. Mai 2024

• References

• 1 Hvid-Jensen F, Pedersen L, Drewes AM. et al. Incidence of adenocarcinoma among patients with Barrett’s esophagus. N Engl J Med 2011; 365: 1375-1383
• 2 Coleman HG, Xie SH, Lagergren J. The epidemiology of esophageal adenocarcinoma. Gastroenterology 2018; 154: 390-405
• 3 Sung H, Ferlay J, Siegel RL. et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2021; 71: 209-249
• 4 Smyth EC, Lagergren J, Fitzgerald RC. et al. Oesophageal cancer. Nat Rev Dis Primers 2017; 3: 17048
• 5 Visrodia K, Singh S, Krishnamoorthi R. et al. Magnitude of missed esophageal adenocarcinoma after Barrett’s esophagus diagnosis: a systematic review and meta-analysis. Gastroenterology 2016; 150: 599-607.e597
• 6 Messmann H, Ebigbo A, Hassan C. et al. How to integrate artificial intelligence in gastrointestinal practice. Gastroenterology 2022; 162: 1583-1586
• 7 van der Sommen F, Zinger S, Curvers WL. et al. Computer-aided detection of early neoplastic lesions in Barrett’s esophagus. Endoscopy 2016; 48: 617-624
• 8 de Groof AJ, Struyvenberg MR, van der Putten J. et al. Deep-learning system detects neoplasia in patients with Barrett’s esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking. Gastroenterology 2020; 158: 915-929.e914
• 9 de Groof AJ, Struyvenberg MR, Fockens KN. et al. Deep learning algorithm detection of Barrett’s neoplasia with high accuracy during live endoscopic procedures: a pilot study (with video). Gastrointest Endosc 2020; 91: 1242-1250
• 10 Hashimoto R, Requa J, Dao T. et al. Artificial intelligence using convolutional neural networks for real-time detection of early esophageal neoplasia in Barrett’s esophagus (with video). Gastrointest Endosc 2020; 91: 1264-1271.e1261
• 11 Iwagami H, Ishihara R, Aoyama K. et al. Artificial intelligence for the detection of esophageal and esophagogastric junctional adenocarcinoma. J Gastroenterol Hepatol 2021; 36: 131-136
• 12 Struyvenberg MR, de Groof AJ, van der Putten J. et al. A computer-assisted algorithm for narrow-band imaging-based tissue characterization in Barrett’s esophagus. Gastrointest Endosc 2021; 93: 89-98
• 13 Hussein M, González-Bueno Puyal J, Lines D. et al. A new artificial intelligence system successfully detects and localises early neoplasia in Barrett’s esophagus by using convolutional neural networks. United European Gastroenterol J 2022; 10: 528-537
• 14 Ebigbo A, Mendel R, Probst A. et al. Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma. Gut 2019; 68: 1143-1145
• 15 Messmann H, Bisschops R, Antonelli G. et al. Expected value of artificial intelligence in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2022; 54: 1211-1231
• 16 Vasey B, Nagendran M, Campbell B. et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med 2022; 28: 924-933
• 17 Chen L-C, Zhu Y, Papandreou G. et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. et al. In: Ferrari V, Hebert M, Sminchisescu C. Computer Vision – ECCV 2018. Cham: Springer International Publishing; 2018: 833-851
• 18 Huang Y, Wang Q, Jia W. et al. See more than once: kernel-sharing atrous convolution for semantic segmentation. Neurocomputing 2021; 443: 26-34
• 19 He K, Zhang X, Ren S. et al. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016; 770-778
• 20 Mendel R, Rauber D, de Souza Jr LA. et al. Error-Correcting Mean-Teacher: corrections instead of consistency-targets applied to semi-supervised medical image segmentation. Comput Biol Med 2023; 154: 106585
• 21 Singer ME, Odze RD. High rate of missed Barrett’s esophagus when screening with forceps biopsies. Esophagus 2023; 20: 143-149
• 22 Ebigbo A, Mendel R, Probst A. et al. Real-time use of artificial intelligence in the evaluation of cancer in Barrett’s oesophagus. Gut 2020; 69: 615-616
• 23 Meinikheim M, Mendel R, Scheppach MW. et al. Influence of an artificial intelligence (AI) based decision support system (DSS) on the diagnostic performance of non-experts in Barrett’s esophagus related neoplasia (BERN). Endoscopy 2022; 54: OP076
• 24 Weusten B, Bisschops R, Coron E. et al. Endoscopic management of Barrett’s esophagus: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2017; 49: 191-198
• 25 Wu L, Shang R, Sharma P. et al. Effect of a deep learning-based system on the miss rate of gastric neoplasms during upper gastrointestinal endoscopy: a single-centre, tandem, randomised controlled trial. Lancet Gastroenterol Hepatol 2021; 6: 700-708
• 26 Glissen Brown JR, Mansour NM, Wang P. et al. Deep learning computer-aided polyp detection reduces adenoma miss rate: a United States multi-center randomized tandem colonoscopy study (CADeT-CS Trial). Clin Gastroenterol Hepatol 2022; 20: 1499-1507.e1494
• 27 Wallace MB, Sharma P, Bhandari P. et al. Impact of artificial intelligence on miss rate of colorectal neoplasia. Gastroenterology 2022; 163: 295-304.e295
• 28 Fockens KN, Jukema JB, Boers T. et al. Towards a robust and compact deep learning system for primary detection of early Barrett’s neoplasia: initial image-based results of training on a multi-center retrospectively collected data set. United European Gastroenterol J 2023; 11: 324-336
• 29 Abdelrahim M, Saiko M, Maeda N. et al. Development and validation of artificial neural networks model for detection of Barrett’s neoplasia: a multicenter pragmatic nonrandomized trial (with video). Gastrointest Endosc 2023; 97: 422-434