May the force be with you: will artificial intelligence take over traditional endoscopy?

Since the introduction of the first fully flexible fiberoptic endoscope by Hirschowitz in 1957 and the advent of digital video endoscopy developed by Welch Allyn in 1984, the gross design and mechanical buildup of flexible endoscopes have not been altered fundamentally. In principle, further improvements have been achieved mainly in the field of image quality, such as resolution, illumination, field of view, magnification, and (virtual) image-enhancement technologies. However, the primary aim of diagnostic procedures is the detection of pathological findings, namely precancerous or early cancerous lesions in the gastrointestinal tract, which is still mainly determined by the ability of the endoscopist to first detect then to correctly characterize a lesion. In this context, we have to admit that seeing a lesion is not equal to recognizing it. Nowadays, clear quality indicators have been established for diagnostic and therapeutic procedures in the gastrointestinal tract, for example, the adenoma detection rate (ADR) in the setting of a screening colonoscopy [1]. We also divide endoscopists examining the lower gastrointestinal tract into high and low detectors. However, quality of diagnostic as well as therapeutic endoscopy is still widely related to the skills of the individual endoscopist. In this context, computer-assisted diagnostic (CAD) systems could be the key to achieving a consistently high(er) level of quality.

“...the novel AI-based assistant system has the potential to help inexperienced endoscopists decrease the risk of inappropriate therapeutic strategies and reduce the risk of adverse events during ERCP.”

Artificial intelligence (AI), first proposed in the 1950 s, uses computers to simulate certain thought processes and forms of intelligent behavior [2]. Machine learning is a branch of AI focusing on computer algorithms that can learn from data and perform specific tasks and analyses [3]. Deep learning is an advanced subset of machine learning that relies on specific algorithms termed artificial neural networks. AI technologies may assist physicians by assimilating and interpreting clinical data to support physician performance [3]. Most of these AI algorithms refer to the field of computer vision, using technologies that allow for recognition and interpretation of visual data (i. e. the live endoscopic image [4]), and fall into two main groups – computer-aided detection (CADe) systems and computer-aided diagnosis (CADx) systems, both of which have already been extensively studied for polyp detection and characterization in colonoscopy [5 – 7]. Among others, further applications in gastrointestinal endoscopy may include automatic assessment of bowel preparation quality or detection of gastric and esophageal (pre-)neoplasms. Beyond endoscopy, computer vision is also being applied quite extensively in abdominal radiology.

However, application of AI technology in gastrointestinal endoscopy may not be limited to computer vision. Further potential applications may be used to reduce physician burden, including automatization of documentation, repetitive and administrative tasks, as well as generation of reports and analytics. Moreover, computer-aided monitoring (CADm) systems have been designed to evaluate the examination procedure and improve the quality of endoscopy (e. g. completeness of mucosal visualization during screening colonoscopy [3]). AI may also be used to facilitate therapeutic procedures in endoscopy; future therapeutic AI algorithms could support physicians in determining the optimal treatment strategy as well as in performing therapeutic procedures (e. g. identification and delineation of tumor margins or labelling anatomical landmarks). A potential advantage of connected AI systems could be that data from electronic health records (e. g. laboratory results, reports of previous endoscopies or radiologic examinations) as well as scientific data from AI-enabled knowledge retrieval tools can be integrated in a clinical decision support system. When basic issues relating to standards for data exchange and protection have been addressed, AI algorithms will be able to automatically store, label, and share images and videos for clinical and scientific application.

The management of common bile duct stones was addressed recently in a European Society of Gastrointestinal Endoscopy guideline [8]. Endoscopic retrograde cholangiopancreatography (ERCP) using conventional treatment strategies such as biliary sphincterotomy and basket or balloon extraction, respectively, achieves stone clearance rates of about 90 % [8]. “Difficult biliary stones” are defined according to their diameter, number, shape, or location, or because of anatomical factors. Clearance of a difficult stone cannot usually be achieved using standard techniques, so multiple procedures and additional interventional techniques (large-balloon dilation, mechanical lithotripsy, cholangioscopy-assisted electrohydraulic/laser lithotripsy, or extracorporeal shock wave lithotripsy) may be required. In this context, the aim of the study by Huang et al. in this issue of Endoscopy was to develop and validate an AI-based difficulty scoring system to predict the complexity of stone treatment [9]. A deep convolutional neuronal network was trained with 1381 cholangiograms and then validated with a test dataset of 573 images. AI detected and characterized size and number of CBD stones with an overall accuracy of 91.45 %. The kappa value for estimation of difficulty of stone extraction was substantial between AI and expert endoscopists, whereas AI was superior to nonexperts. With respect to clinical outcomes, stone clearance rates using standard techniques were 86.26 % and 36.36 % in cases for which AI predicted low and high difficulty, respectively. In conclusion, on the basis of this study, the novel AI-based assistant system has the potential to help inexperienced endoscopists decrease the risk of inappropriate therapeutic strategies and reduce the risk of adverse events during ERCP.

We can expect to see a fast development of AI technology in all sectors of gastrointestinal endoscopy in the coming years. While integration of AI will fundamentally change efficacy, safety, and workflow of direct patient care, new systems will be able to reduce physicians’ burden of administrative work as well.

In consequence, AI will be integrated into gastrointestinal endoscopy in various fields and will empower each and every single endoscopist with nothing less than the knowledge and experience of the entire community of endoscopists, and will therefore make quality of endoscopy more independent from the individual physician.

However, before we can reach this goal, each of us will have to trust the “new force” that has been handed to us by computer scientists before the future is ready to take over traditional endoscopy. In addition, more studies are needed to obtain higher quality data to build this trust. We as endoscopists have to be open-minded, accept the challenges that are posed by this development, and be willing to explore these new horizons. Finally, it is good to know that we are no longer alone in the endoscopy theater – trust in the force and may the force be with us!

Publication History

Article published online:
22 April 2021

• References

• 1 Rex DK, Petrini JL, Baron TH. et al. Quality indicators for colonoscopy. Am J Gastroenterol 2006; 101: 873-885
• 2 He YS, Su JR, Li Z. et al. Application of artificial intelligence in gastrointestinal endoscopy. J Dig Dis 2019; 20: 623-630
• 3 Berzin TM, Parasa S, Wallace MB. et al. Position statement on priorities for artificial intelligence in GI endoscopy: a report by the ASGE Task Force. Gastrointest Endosc 2020; 92: 951-959
• 4 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436-444
• 5 Byrne MF, Chapados N, Soudan F. et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2019; 68: 94-100
• 6 Repici A, Badalamenti M, Maselli R. et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520
• 7 Wang P, Berzin TM, Glissen Brown JR. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68: 1813-1819
• 8 Manes G, Paspatis G, Aabakken L. et al. Endoscopic management of common bile duct stones: European Society of Gastrointestinal Endoscopy (ESGE) guideline. Endoscopy 2019; 51: 472-491
• 9 Huang L, Lu X, Huang X. et al. Intelligent difficulty scoring and assistance system for endoscopic extraction of common bile duct stones based on deep learning: multicenter study. Endoscopy 2020; 491-498