Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method

Omer F. Ahmad; Yuichi Mori; Masashi Misawa; Shin-ei Kudo; John T. Anderson; Jorge Bernal; Tyler M. Berzin; Raf Bisschops; Michael F. Byrne; Peng-Jen Chen; James E. East; Tom Eelbode; Daniel S. Elson; Suryakanth R. Gurudu; Aymeric Histace; William E. Karnes; Alessandro Repici; Rajvinder Singh; Pietro Valdastri; Michael B. Wallace; Pu Wang; Danail Stoyanov; Laurence B. Lovat

doi:10.1055/a-1306-7590

Endoscopy, Table of Contents

CC BY 4.0 · Endoscopy 2021; 53(09): 893-901
DOI: 10.1055/a-1306-7590

Original article

Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method

Omer F. Ahmad

¹Wellcome/EPSRC Centre for Interventional & Surgical Sciences, University College London, London, UK

,

Yuichi Mori

²Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan

³Clinical Effectiveness Research Group, Institute of Health and Society, University of Oslo, Oslo, Norway

,

Masashi Misawa

²Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan

,

Shin-ei Kudo

²Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan

,

John T. Anderson

⁴Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, UK

,

Jorge Bernal

⁵Computer Science Department, Universitat Autonoma de Barcelona and Computer Vision Center, Barcelona, Spain

,

Tyler M. Berzin

⁶Center for Advanced Endoscopy, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA

,

Raf Bisschops

⁷Department of Gastroenterology and Hepatology, University Hospitals Leuven, TARGID KU Leuven, Leuven, Belgium

,

Michael F. Byrne

⁸Division of Gastroenterology, Vancouver General Hospital, University of British Columbia, Vancouver, British Columbia, Canada

,

Peng-Jen Chen

⁹Division of Gastroenterology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan

,

James E. East

¹⁰Translational Gastroenterology Unit, John Radcliffe Hospital, Oxford, UK

¹¹Oxford NIHR Biomedical Research Centre, University of Oxford, Oxford, UK

,

Tom Eelbode

¹²Medical Imaging Research Center, ESAT/PSI, KU Leuven, Leuven, Belgium

,

Daniel S. Elson

¹³Hamlyn Centre for Robotic Surgery, Institute of Global Health Innovation, Imperial College London, London, UK

¹⁴Department of Surgery and Cancer, Imperial College London, London, UK

,

Suryakanth R. Gurudu

¹⁵Division of Gastroenterology and Hepatology, Mayo Clinic, Scottsdale, Arizona, USA

,

Aymeric Histace

¹⁶ETIS, Universite de Cergy-Pointoise, ENSEA, CNRS, Cergy-Pointoise Cedex, France

,

William E. Karnes

¹⁷H. H. Chao Comprehensive Digestive Disease Center, Division of Gastroenterology & Hepatology, Department of Medicine, University of California, Irvine, California, USA

,

Alessandro Repici

¹⁸Department of Gastroenterology, Humanitas Clinical and Research Center, IRCCS, Rozzano, Milan, Italy

¹⁹Humanitas University, Department of Biomedical Sciences, Pieve Emanuele, Milan, Italy

,

Rajvinder Singh

²⁰Department of Gastroenterology and Hepatology, Lyell McEwan Hospital, Adelaide, South Australia, Australia

,

Pietro Valdastri

²¹School of Electronics and Electrical Engineering, University of Leeds, Leeds, UK

,

Michael B. Wallace

²²Division of Gastroenterology & Hepatology, Mayo Clinic, Jacksonville, Florida, USA

,

Pu Wang

²³Department of Gastroenterology, Sichuan Academy of Medical Sciences and Sichuan Provincial People’s Hospital, Chengdu, China

,

Danail Stoyanov

¹Wellcome/EPSRC Centre for Interventional & Surgical Sciences, University College London, London, UK

,

Laurence B. Lovat

¹Wellcome/EPSRC Centre for Interventional & Surgical Sciences, University College London, London, UK

²⁴Gastrointestinal Services, University College London Hospital, London, UK

› Author Affiliations

Abstract

Background Artificial intelligence (AI) research in colonoscopy is progressing rapidly but widespread clinical implementation is not yet a reality. We aimed to identify the top implementation research priorities.

Methods An established modified Delphi approach for research priority setting was used. Fifteen international experts, including endoscopists and translational computer scientists/engineers, from nine countries participated in an online survey over 9 months. Questions related to AI implementation in colonoscopy were generated as a long-list in the first round, and then scored in two subsequent rounds to identify the top 10 research questions.

Results The top 10 ranked questions were categorized into five themes. Theme 1: clinical trial design/end points (4 questions), related to optimum trial designs for polyp detection and characterization, determining the optimal end points for evaluation of AI, and demonstrating impact on interval cancer rates. Theme 2: technological developments (3 questions), including improving detection of more challenging and advanced lesions, reduction of false-positive rates, and minimizing latency. Theme 3: clinical adoption/integration (1 question), concerning the effective combination of detection and characterization into one workflow. Theme 4: data access/annotation (1 question), concerning more efficient or automated data annotation methods to reduce the burden on human experts. Theme 5: regulatory approval (1 question), related to making regulatory approval processes more efficient.

Conclusions This is the first reported international research priority setting exercise for AI in colonoscopy. The study findings should be used as a framework to guide future research with key stakeholders to accelerate the clinical implementation of AI in endoscopy.

Full Text

References

References
1 Kudo S, Mori Y, Misawa M. et al. Artificial intelligence and colonoscopy: Current status and future perspectives. Dig Endosc 2019; 31: 363-371
2 Ahmad OF, Soares AS, Mazomenos E. et al. Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions. Lancet Gastroenterol Hepatol 2019; 4: 71-80
3 Mori Y, Kudo S, Berzin TM. et al. Computer-aided diagnosis for colonoscopy. Endoscopy 2017; 49: 813-819
4 Wang P, Berzin TM, Glissen Brown JR. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68: 1813-1819
5 Wang P, Liu X, Berzin TM. et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol 2020; 5: 343-351
6 Su J-R, Li Z, Shao X-J. et al. Impact of a real-time automatic quality control system on colorectal polyp and adenoma detection: a prospective randomized controlled study (with videos). Gastrointest Endosc 2020; 91: 415-424.e4
7 Liu W-N, Zhang Y-Y, Bian X-Q. et al. Study on detection rate of polyps and adenomas in artificial-intelligence-aided colonoscopy. Saudi J Gastroenterol 2020; 26: 13-19
8 Repici A, Badalamenti M, Maselli R. et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520.e7
9 He J, Baxter SL, Xu J. et al. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019; 25: 30-36
10 Langlotz CP, Allen B, Erickson BJ. et al. A roadmap for foundational research on artificial intelligence in medical imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291: 781-791
11 Vinsard DG, Mori Y, Misawa M. et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest Endosc 2019; 90: 55-63
12 Rees CJ, Ngu WS, Regula J. et al. European Society of Gastrointestinal Endoscopy – Establishing the key unanswered research questions within gastrointestinal endoscopy. Endoscopy 2016; 48: 884-891
13 Francis N, Kazaryan AM, Pietrabissa A. et al. A research agenda for the European Association for Endoscopic Surgeons (EAES). Surg Endosc 2017; 31: 2042-2049
14 Burt CG, Cima RR, Koltun WA. et al. Developing a research agenda for the American Society of Colon and Rectal Surgeons: Results of a Delphi approach. Dis Colon Rectum 2009; 52: 898-905
15 Hart AL, Lomer M, Verjee A. et al. What are the top 10 research questions in the treatment of inflammatory bowel disease? A priority setting partnership with the James Lind Alliance. J Crohns Colitis 2017; 11: 204-211
16 Liu X, Cruz Rivera S, Moher D. et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med 2020; 26: 1364-1374
17 Wang P, Liu P, Glissen Brown JR. et al. Lower adenoma miss rate of computer-aided detection-assisted colonoscopy vs routine white-light colonoscopy in a prospective tandem study. Gastroenterology 2020; 159: 1252-1261.e5
18 Mori Y, Kudo S, Misawa M. et al. Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: A prospective study. Ann Intern Med 2018; 169: 357-366
19 Hassan C, Repici A. The FUSE enigma: Wide-angle or wide-minded?. Gastrointest Endosc 2018; 88: 865-867
20 East JE, Rees CJ. Making optical biopsy a clinical reality in colonoscopy. Lancet Gastroenterol Hepatol 2018; 3: 10-12
21 Bisschops R, East JE, Hassan C. et al. Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline – Update 2019. Endoscopy 2019; 51: 1155-1179
22 Barua I, Vinsard D, Jodal H. et al. Artificial intelligence for polyp detection during colonoscopy: a systematic review and meta-analysis. Endoscopy 2020;
23 Hassan C, Bhandari P, Antonelli G. et al. Artificial intelligence for non-polypoid colorectal neoplasms. Dig Endosc 2020;
24 Bernal J, Tajkbaksh N, Sanchez FJ. et al. Comparative validation of polyp detection methods in video colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge. IEEE Trans Med Imaging 2017; 36: 1231-1249
25 Xu S, Perez M, Yang K. et al. Determination of the latency effects on surgical performance and the acceptable latency levels in telesurgery using the dV-Trainer simulator. Surg Endosc 2014; 28: 2569-2576
26 Byrne MF, Chapados N, Soudan F. et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2019; 68: 94-100
27 Mori Y, Kudo S-E, Misawa M. et al. Simultaneous detection and characterization of diminutive polyps with the use of artificial intelligence during colonoscopy. VideoGIE 2019; 4: 7-10
28 Guizard N, Ghalehjegh SH, Henkel M. et al. Artificial intelligence for real-time multiple polyp detection with identification, tracking, and optical biopsy during colonoscopy. Gastroenterology 2019; 156: S48-S49
29 Sánchez-Montes C, Sánchez FJ, Bernal J. et al. Computer-aided prediction of polyp histology on white light colonoscopy using surface pattern analysis. Endoscopy 2019; 51: 261-265
30 Lutnick B, Ginley B, Govind D. et al. An integrated iterative annotation technique for easing neural network training in medical image analysis. Nat Mach Intell 2019; 1: 112-119
31 Borgli H, Thambawita V, Smedsrud PH. et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data 2020; 7: 283
32 Ahmad OF, Stoyanov D, Lovat LB. Barriers and pitfalls for artificial intelligence in gastroenterology: Ethical and regulatory issues. Tech Innov Gastrointest Endosc 2020; 22: 80-84
33 Mori Y, Kudo S, East JE. et al. Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video). Gastrointest Endosc 2020; 92: 905-911.e1

Supplementary Material

Supplementary material