Download PDF
Endoscopy 2022; 54(04): 403-411
DOI: 10.1055/a-1500-3730
Systematic Review

Endoscopistsʼ diagnostic accuracy in detecting upper gastrointestinal neoplasia in the framework of artificial intelligence studies

Leonardo Frazzoni  ‡   *
 1   Department of Medical and Surgical Sciences (DIMEC), University of Bologna, S. Orsola-Malpighi Hospital, Bologna, Italy
,
Julia Arribas
 2   CIDES/CINTESIS, Faculty of Medicine, University of Porto, Porto, Portugal
,
Giulio Antonelli‡   *
 3   Gastroenterology Unit, Nuovo Regina Margherita Hospital, Rome, Italy
 4   Department of Translational and Precision Medicine, “Sapienza” University of Rome, Rome, Italy
,
Diogo Libanio 
 2   CIDES/CINTESIS, Faculty of Medicine, University of Porto, Porto, Portugal
,
Alanna Ebigbo
 5   III Medizinische Klinik, Universitatsklinikum Augsburg, Augsburg, Germany
,
Fons van der Sommen 
 6   Department of Electrical Engineering, VCA group, Eindhoven University of Technology, Eindhoven, The Netherlands
,
Albert Jeroen de Groof
 7   Department of Gastroenterology and Hepatology, Amsterdam UMC, Amsterdam, The Netherlands
,
Hiromu Fukuda
 8   Department of Gastrointestinal Oncology, Osaka International Cancer Institute, Osaka, Japan
,
Masayasu Ohmori 
 8   Department of Gastrointestinal Oncology, Osaka International Cancer Institute, Osaka, Japan
,
Ryu Ishihara
 8   Department of Gastrointestinal Oncology, Osaka International Cancer Institute, Osaka, Japan
,
Lianlian Wu
 9   Department of Gastroenterology, Renmin Hospital of Wuhan University, Institute for Gastroenterology and Hepatology, Wuhan University, Wuhan, China
,
Honggang Yu 
 9   Department of Gastroenterology, Renmin Hospital of Wuhan University, Institute for Gastroenterology and Hepatology, Wuhan University, Wuhan, China
,
Yuichi Mori
10   Clinical Effectiveness Research Group, University of Oslo, Oslo, Norway
11   Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
,
Alessandro Repici
12   Digestive Endoscopy Unit, Humanitas Research Hospital – IRCCS, Milan, Italy
13   Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Milan, Italy
,
Jacques J. G. H. M. Bergman
 7   Department of Gastroenterology and Hepatology, Amsterdam UMC, Amsterdam, The Netherlands
,
Prateek Sharma
14   Department of Gastroenterology and Hepatology, University of Kansas Medical Center, Kansas City, Kansas, USA
,
Helmut Messmann
 5   III Medizinische Klinik, Universitatsklinikum Augsburg, Augsburg, Germany
,
Cesare Hassan
 3   Gastroenterology Unit, Nuovo Regina Margherita Hospital, Rome, Italy
,
Lorenzo Fuccio 
 1   Department of Medical and Surgical Sciences (DIMEC), University of Bologna, S. Orsola-Malpighi Hospital, Bologna, Italy
,
Mário Dinis-Ribeiro
15   Gastroenterology Department, Portuguese Oncology Institute of Porto, Porto, Portugal
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints

Referred to by:

Author commentary on Leonardo Frazzoni et al.Endoscopy 2022; 54(04): v17-v17
DOI: 10.1055/a-1540-3373

Corrected by:

Correction: Endoscopistsʼ diagnostic accuracy in detecting upper gastrointestinal neoplasia in the framework of artificial intelligence studiesEndoscopy 2022; 54(04): C12-C12
DOI: 10.1055/a-1907-9242

Abstract

Background Estimates on miss rates for upper gastrointestinal neoplasia (UGIN) rely on registry data or old studies. Quality assurance programs for upper GI endoscopy are not fully established owing to the lack of infrastructure to measure endoscopists’ competence. We aimed to assess endoscopists’ accuracy for the recognition of UGIN exploiting the framework of artificial intelligence (AI) validation studies.

Methods Literature searches of databases (PubMed/MEDLINE, EMBASE, Scopus) up to August 2020 were performed to identify articles evaluating the accuracy of individual endoscopists for the recognition of UGIN within studies validating AI against a histologically verified expert-annotated ground-truth. The main outcomes were endoscopists’ pooled sensitivity, specificity, positive and negative predictive value (PPV/NPV), and area under the curve (AUC) for all UGIN, for esophageal squamous cell neoplasia (ESCN), Barrett esophagus-related neoplasia (BERN), and gastric adenocarcinoma (GAC).

Results Seven studies (2 ESCN, 3 BERN, 1 GAC, 1 UGIN overall) with 122 endoscopists were included. The pooled endoscopists’ sensitivity and specificity for UGIN were 82 % (95 % confidence interval [CI] 80 %–84 %) and 79 % (95 %CI 76 %–81 %), respectively. Endoscopists’ accuracy was higher for GAC detection (AUC 0.95 [95 %CI 0.93–0.98]) than for ESCN (AUC 0.90 [95 %CI 0.88–0.92]) and BERN detection (AUC 0.86 [95 %CI 0.84–0.88]). Sensitivity was higher for Eastern vs. Western endoscopists (87 % [95 %CI 84 %–89 %] vs. 75 % [95 %CI 72 %–78 %]), and for expert vs. non-expert endoscopists (85 % [95 %CI 83 %–87 %] vs. 71 % [95 %CI 67 %–75 %]).

Conclusion We show suboptimal accuracy of endoscopists for the recognition of UGIN even within a framework that included a higher prevalence and disease awareness. Future AI validation studies represent a framework to assess endoscopist competence.

* Joint first authors


Appendices 1s–3s, Figs. 1s–6s, Tables 1s–3s



Publication History

Received: 05 January 2021

Accepted after revision: 05 May 2021

Accepted Manuscript online:
05 May 2021

Article published online:
17 June 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Arnold M, Abnet CC, Neale RE. et al. Global burden of 5 major types of gastrointestinal cancer. Gastroenterology 2020; 159: 335-349.e15
    CrossrefPubMedSearch in Google Scholar
  • 2 Pimentel-Nunes P, Libânio D, Marcos-Pinto R. et al. Management of epithelial precancerous conditions and lesions in the stomach (MAPS II): European Society of Gastrointestinal Endoscopy (ESGE), European Helicobacter and Microbiota Study Group (EHMSG), European Society of Pathology (ESP), and Sociedade Portuguesa de Endoscopia Digestiva (SPED) guideline update 2019. Endoscopy 2019; 51: 365-388
    Search in Google Scholar
  • 3 Areia M, Spaander MC, Kuipers EJ. et al. Endoscopic screening for gastric cancer: A cost-utility analysis for countries with an intermediate gastric cancer risk. United European Gastroenterol J 2018; 6: 192-202
    CrossrefPubMedSearch in Google Scholar
  • 4 Areia M, Carvalho R, Cadime AT. et al. Screening for gastric cancer and surveillance of premalignant lesions: a systematic review of cost-effectiveness studies. Helicobacter 2013; 18: 325-337
    CrossrefPubMedSearch in Google Scholar
  • 5 di Pietro M, Chan D, Fitzgerald RC. et al. Screening for Barrett’s esophagus. Gastroenterology 2015; 148: 912-923
    CrossrefPubMedSearch in Google Scholar
  • 6 Săftoiu A, Hassan C, Areia M. et al. Role of gastrointestinal endoscopy in the screening of digestive tract cancers in Europe: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2020; 52: 293-304
    Thieme ConnectPubMedSearch in Google Scholar
  • 7 Pimenta-Melo AR, Monteiro-Soares M, Libânio D. et al. Missing rate for gastric cancer during upper gastrointestinal endoscopy: a systematic review and meta-analysis. Eur J Gastroenterol Hepatol 2016; 28: 1041-1049
    CrossrefPubMedSearch in Google Scholar
  • 8 Hernanz N, Rodríguez de Santiago E, Marcos Prieto HM. et al. Characteristics and consequences of missed gastric cancer: A multicentric cohort study. Dig Liver Dis 2019; 51: 894-900
    CrossrefPubMedSearch in Google Scholar
  • 9 Rodríguez de Santiago E, Hernanz N, Marcos-Prieto HM. et al. Rate of missed oesophageal cancer at routine endoscopy and survival outcomes: A multicentric cohort study. United European Gastroenterol J 2019; 7: 189-198
    CrossrefPubMedSearch in Google Scholar
  • 10 Kaminski MF, Thomas-Gibson S, Bugajski M. et al. Performance measures for lower gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative. Endoscopy 2017; 49: 378-397
    Search in Google Scholar
  • 11 van der Sommen F, de Groof J, Struyvenberg M. et al. Machine learning in GI endoscopy: practical guidance in how to interpret a novel field. Gut 2020; 69: 2035-2045
    CrossrefPubMedSearch in Google Scholar
  • 12 Arribas J, Antonelli G, Frazzoni L. et al. Standalone performance of artificial intelligence for upper GI neoplasia: a meta-analysis. Gut 2020;
    CrossrefPubMedSearch in Google Scholar
  • 13 Bisschops R, Areia M, Coron E. et al. Performance measures for upper gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative. Endoscopy 2016; 48: 843-864
    Search in Google Scholar
  • 14 Moher D, Liberati A, Tetzlaff J. et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009; 62: 1006-1012
    Search in Google Scholar
  • 15 Whiting PF, Rutjes AWS, Westwood ME. et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155: 529-536
    Search in Google Scholar
  • 16 Reitsma JB, Glas AS, Rutjes AWS. et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005; 58: 982-990
    CrossrefPubMedSearch in Google Scholar
  • 17 Naaktgeboren CA, Ochodo EA, Van Enst WA. et al. Assessing variability in results in systematic reviews of diagnostic studies. BMC Med Res Methodol 2016; 16: 6
    CrossrefPubMedSearch in Google Scholar
  • 18 Doebler P, Holling H. Meta-analysis of diagnostic accuracy with mada. https://rdrr.io/rforge/mada/f/inst/doc/mada.pdf Accessed: 7 May 2021.
    PubMed
  • 19 R: a language and environment for statistical computing. https://www.gbif.org/tool/81287/r-a-language-and-environment-for-statistical-computing Accessed: 7 May 2021.
    PubMed
  • 20 Horiuchi Y, Aoyama K, Tokai Y. et al. Convolutional neural network for differentiating gastric cancer from gastritis using magnified endoscopy with narrow band imaging. Dig Dis Sci 2020; 65: 1355-1363
    CrossrefPubMedSearch in Google Scholar
  • 21 Ikenoyama Y, Hirasawa T, Ishioka M. et al. Detecting early gastric cancer: Comparison between the diagnostic ability of convolutional neural networks and endoscopists. Dig Endosc 2021; 33: 141-150
    CrossrefPubMedSearch in Google Scholar
  • 22 Cai S-L, Li B, Tan W-M. et al. Using a deep learning system in endoscopy for screening of early esophageal squamous cell carcinoma (with video). Gastrointest Endosc 2019; 90: 745-753.e2
    CrossrefPubMedSearch in Google Scholar
  • 23 Ohmori M, Ishihara R, Aoyama K. et al. Endoscopic detection and differentiation of esophageal lesions using a deep neural network. Gastrointest Endosc 2020; 91: 301-309.e1
    CrossrefPubMedSearch in Google Scholar
  • 24 Fukuda H, Ishihara R, Kato Y. et al. Comparison of performances of artificial intelligence versus expert endoscopists for real-time assisted diagnosis of esophageal squamous cell carcinoma (with video). Gastrointest Endosc 2020; 92: 848-855
    CrossrefPubMedSearch in Google Scholar
  • 25 Ebigbo A, Mendel R, Probst A. et al. Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma. Gut 2019; 68: 1143-1145
    CrossrefPubMedSearch in Google Scholar
  • 26 de Groof AJ, Struyvenberg MR, van der Putten J. et al. Deep-learning system detects neoplasia in patients with Barrett’s esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking. Gastroenterology 2020; 158: 915-929.e4
    CrossrefPubMedSearch in Google Scholar
  • 27 van der Sommen F, Zinger S, Curvers W. et al. Computer-aided detection of early neoplastic lesions in Barrett’s esophagus. Endoscopy 2016; 48: 617-624
    Thieme ConnectPubMedSearch in Google Scholar
  • 28 Wu L, Zhou W, Wan X. et al. A deep neural network improves endoscopic detection of early gastric cancer without blind spots. Endoscopy 2019; 51: 522-531
    Thieme ConnectPubMedSearch in Google Scholar
  • 29 Luo H, Xu G, Li C. et al. Real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy: a multicentre, case-control, diagnostic study. Lancet Oncol 2019; 20: 1645-1654
    Search in Google Scholar
  • 30 Uedo N, Gotoda T, Yoshinaga S. et al. Differences in routine esophagogastroduodenoscopy between Japanese and international facilities: A questionnaire survey. Dig Endosc 2016; 28: 16-24
    CrossrefPubMedSearch in Google Scholar