How can we improve our operation?

We read the study conducted by Zippelius et al. [1] with great interest and thank the authors for revealing a comparable noninferior performance between an artificial intelligence (AI) system and endoscopists. Although the test in a hospital independent of the system development provides validation evidence, unlike other high-quality trials with larger sample sizes and different clinic levels, this research did not detect different performance [2] [3] [4] [5]. To maximize the confidence of the comparison, we would like to get a closer look at the study and make some comments.

1. Without sample size estimation, the number of patients assigned to some endoscopists appeared too small, which might blur any performance decline due to distraction and fatigue caused by long working hours and may make it difficult to compare the performance between endoscopists and AI. Furthermore, all sessile serrated polyps were found by more experienced endoscopists and significant differences in detection rates could be observed. Therefore, we recommend dividing endoscopists into groups based on level of experience and increasing the sample size in each subgroup for further evaluation.

2. During the endoscopy procedures, some transient detections by the AI system might emerge when folds are flattened by the endoscopist for sample taking. Authors simply regarded the green squares as the AI outcome, but did not define the exclusion criteria for transient detection, especially the additional time involved for endoscopists’ observation and manipulation, which may underestimate the miss rate of AI.

3. The endoscopist’s verbal indication of a lesion while not focusing on the lesion will decrease the observation time of endoscopists. Furthermore, keeping the AI system in operation during resection could also give AI an advantage.

4. We recommend showing images of the two lesions missed by AI, considering that false alarms including vascular structures, mucosal lesions and inflammation might be signs of early dysplasia.

Our concerns should not dissuade readers from considering the importance of this article, but we hope our suggestions could improve the strength of evidence in follow-up studies.

Publication History

Article published online:
27 January 2022

• References

• 1 Zippelius C, Alqahtani SA, Schedel J. et al. Diagnostic accuracy of a novel artificial intelligence system for adenoma detection in daily practice: a prospective nonrandomized comparative study. Endoscopy 2021; DOI: 10.1055/a-1556-5984.
• 2 Wu L, He X, Liu M. et al. Evaluation of the effects of an artificial intelligence system on endoscopy quality and preliminary testing of its performance in detecting early gastric cancer: a randomized controlled trial. Endoscopy 2021; DOI: 10.1055/a-1350-5583.
• 3 Wu L, Shang R, Sharma P. et al. Effect of a deep learning-based system on the miss rate of gastric neoplasms during upper gastrointestinal endoscopy: a single-centre, tandem, randomised controlled trial. Lancet Gastroenterol Hepatol 2021; 6: 700-708
• 4 Repici A, Badalamenti M, Maselli R. et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520
• 5 Repici A, Spadaccini M, Antonelli G. et al. Artificial intelligence and colonoscopy experience: lessons from two randomised trials. Gut 2021; DOI: 10.1136/gutjnl-2021-324471.