Download PDF
CC BY 4.0 · Endoscopy
DOI: 10.1055/a-2530-1845
Original article

Impact of standard enhancement settings of endoscopy systems on performance of endoscopic artificial intelligence systems

Martijn R. Jong‡   ‡
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Carolus H. J. Kusters‡   ‡
2   Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
,
Querijn N. E. van Bokhorst
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Jelmer B. Jukema
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Rixta A. H. van Eijck van Heslinga
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Kiki N. Fockens
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Britt B. S. L. Houwen
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Tim J. M. Jaspers
2   Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
,
Tim G. W. Boers
2   Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
,
Manon van der Vlugt
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Evelien Dekker 
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Fons van der Sommen
2   Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
,
Peter H. N. de With
2   Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
,
Albert J. de Groof
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
Jacques J. G. H. M Bergman 
1   Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
,
on behalf of the BONS-AI Consortium › Author AffiliationsSupported by: Nederlandse Organisatie voor Wetenschappelijk Onderzoek http://dx.doi.org/10.13039/501100003246
Supported by: DANAE Project supported by NWO/KWF
› Further Information
Also available at
   Permissions and Reprints


Abstract

Background Artificial intelligence (AI) systems in endoscopy are predominantly developed and tested using high-quality imagery from expert centers. However, their performance may be different when applied in clinical practice, partly due to the diversity in post-processing enhancement settings used in endoscopy units. We evaluated the impact of post-processing enhancement settings on AI performance and tested specific data augmentation strategies to mitigate performance loss.

Methods We used a computer-aided detection (CADe) system for Barrett’s neoplasia (6223 images, 906 patients) and a computer-aided diagnosis (CADx) system for colorectal polyps (3288 images, 969 patients), both trained on datasets acquired with Olympus equipment and with limited variability in enhancement settings. The CAD systems were then tested across a wide range of test sets, which comprised the same images, but displayed with different enhancement settings. Both CAD systems were then retrained using image enhancement-based data augmentation. The performance of the adjusted CAD systems was evaluated on the same test sets.

Results Both systems displayed substantial performance variability over a range of enhancement settings (CADe: 83 %–92 % sensitivity, 84 %–91 % specificity; CADx: 78 %–85 % sensitivity, 45 %–63 % specificity). After retraining, variability in sensitivity and specificity was reduced to 2 % (P < 0.001) and 1 %, respectively (P = 0.003) for CADe, and 2 % (P = 0.03) and 8 %, respectively (P = 0.19) for CADx.

Conclusion The performance of endoscopic AI systems can vary substantially depending on post-processing enhancement settings of the endoscopy unit. Specific data augmentation can mitigate this performance loss.

‡ Co-first authors.


* All members and collaborators of the Barrett’s Oesophagus Imaging for Artificial Intelligence (BONS-AI) Consortium are listed in the Supplementary material.


Supplementary Material



Publication History

Received: 03 September 2024

Accepted after revision: 29 January 2025

Accepted Manuscript online:
30 January 2025

Article published online:
28 February 2025

© 2025. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany

 

  • References

  • 1 Wallace MB, Sharma P, Bhandari P. et al. Impact of artificial intelligence on miss rate of colorectal neoplasia. Gastroenterology 2022; 163: 295-304.e5
    CrossrefPubMedSearch in Google Scholar
  • 2 Fockens N, Jong MR, Jukema JB. et al. A deep learning system for detection of early Barrett’s neoplasia: a model development and validation study. Lancet Digit Health 2023; 5: 905-921
    CrossrefPubMedSearch in Google Scholar
  • 3 Finlayson SG, Subbaswamy A, Singh K. et al. The clinician and dataset shift in artificial intelligence. New Engl J Med 2021; 385: 283-286
    CrossrefPubMedSearch in Google Scholar
  • 4 Subbaswamy A, Saria S. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics 2020; 21: 345-352
    PubMedSearch in Google Scholar
  • 5 DeGrave AJ, Janizek JD, Lee SI. AI for radiographic COVID-19 detection selects shortcuts over signal. Nat Mach Intell 2021; 3: 610-619
    CrossrefPubMedSearch in Google Scholar
  • 6 Ugurlu D, Puyol-Anton E, Ruijsink B. et al. The impact of domain shift on left and right ventricle segmentation in short axis cardiac MR images. arXiv [Preprint] 2021;
    CrossrefPubMedSearch in Google Scholar
  • 7 Pooch EHP, Ballester PL, Barros RC. Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classification. arXiv [Preprint] 2019;
    CrossrefPubMedSearch in Google Scholar
  • 8 Perez L, Wang J. The effectiveness of data augmentation in image classification using deep learning. arXiv [Preprint] 2017;
    CrossrefPubMedSearch in Google Scholar
  • 9 Fockens KN, Jukema JB, Boers T. et al. Towards a robust and compact deep learning system for primary detection of early Barrett’s neoplasia: initial image-based results of training on a multi-center retrospectively collected data set. United European Gastroenterol J 2023; 11: 324-336
    CrossrefPubMedSearch in Google Scholar
  • 10 Houwen BB, Hazewinkel Y, Giotis I. et al. Real-time computer-aided diagnosis system for optical diagnosis of diminutive colorectal polyps including sessile serrated lesions: a prospective, multicenter study with benchmarking against screening endoscopists. Endoscopy 2022; 54: eP133
    PubMedSearch in Google Scholar
  • 11 Yu AC, Mohajer B, Eng J. External validation of deep learning algorithms for radiologic diagnosis: a systematic review. Radiol Artif Intell 2022; 4: e210064
    CrossrefPubMedSearch in Google Scholar
  • 12 Karsenti D, Tharsis G, Perrot B. et al. Effect of real-time computer-aided detection of colorectal adenoma in routine colonoscopy (COLO-GENIUS): a single-centre randomised controlled trial. Lancet Gastroenterol Hepatol 2023; 8: 726-734
    CrossrefPubMedSearch in Google Scholar
  • 13 Levy I, Bruckmayer L, Klang E. et al. Artificial intelligence-aided colonoscopy does not increase adenoma detection rate in routine clinical practice. Am J Gastroenterol 2022; 117: 1871-1873
    CrossrefPubMedSearch in Google Scholar