Download PDF
CC BY 4.0 · Endoscopy 2021; 53(09): 893-901
DOI: 10.1055/a-1306-7590
Original article

Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method

Omer F. Ahmad 
 1   Wellcome/EPSRC Centre for Interventional & Surgical Sciences, University College London, London, UK
,
Yuichi Mori 
 2   Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
 3   Clinical Effectiveness Research Group, Institute of Health and Society, University of Oslo, Oslo, Norway
,
Masashi Misawa
 2   Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
,
Shin-ei Kudo 
 2   Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
,
John T. Anderson 
 4   Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, UK
,
Jorge Bernal
 5   Computer Science Department, Universitat Autonoma de Barcelona and Computer Vision Center, Barcelona, Spain
,
Tyler M. Berzin
 6   Center for Advanced Endoscopy, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
,
Raf Bisschops 
 7   Department of Gastroenterology and Hepatology, University Hospitals Leuven, TARGID KU Leuven, Leuven, Belgium
,
Michael F. Byrne
 8   Division of Gastroenterology, Vancouver General Hospital, University of British Columbia, Vancouver, British Columbia, Canada
,
Peng-Jen Chen 
 9   Division of Gastroenterology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
,
James E. East 
10   Translational Gastroenterology Unit, John Radcliffe Hospital, Oxford, UK
11   Oxford NIHR Biomedical Research Centre, University of Oxford, Oxford, UK
,
Tom Eelbode
12   Medical Imaging Research Center, ESAT/PSI, KU Leuven, Leuven, Belgium
,
Daniel S. Elson
13   Hamlyn Centre for Robotic Surgery, Institute of Global Health Innovation, Imperial College London, London, UK
14   Department of Surgery and Cancer, Imperial College London, London, UK
,
Suryakanth R. Gurudu
15   Division of Gastroenterology and Hepatology, Mayo Clinic, Scottsdale, Arizona, USA
,
Aymeric Histace 
16   ETIS, Universite de Cergy-Pointoise, ENSEA, CNRS, Cergy-Pointoise Cedex, France
,
William E. Karnes
17   H. H. Chao Comprehensive Digestive Disease Center, Division of Gastroenterology & Hepatology, Department of Medicine, University of California, Irvine, California, USA
,
Alessandro Repici
18   Department of Gastroenterology, Humanitas Clinical and Research Center, IRCCS, Rozzano, Milan, Italy
19   Humanitas University, Department of Biomedical Sciences, Pieve Emanuele, Milan, Italy
,
Rajvinder Singh
20   Department of Gastroenterology and Hepatology, Lyell McEwan Hospital, Adelaide, South Australia, Australia
,
Pietro Valdastri
21   School of Electronics and Electrical Engineering, University of Leeds, Leeds, UK
,
Michael B. Wallace 
22   Division of Gastroenterology & Hepatology, Mayo Clinic, Jacksonville, Florida, USA
,
Pu Wang 
23   Department of Gastroenterology, Sichuan Academy of Medical Sciences and Sichuan Provincial People’s Hospital, Chengdu, China
,
Danail Stoyanov
 1   Wellcome/EPSRC Centre for Interventional & Surgical Sciences, University College London, London, UK
,
Laurence B. Lovat
 1   Wellcome/EPSRC Centre for Interventional & Surgical Sciences, University College London, London, UK
24   Gastrointestinal Services, University College London Hospital, London, UK
› Author AffiliationsSupported by: Wellcome Trust 203145Z/16/Z
Supported by: Engineering and Physical Sciences Research Council 203145Z/16/Z
Supported by: Engineering and Physical Sciences Research Council EP/P027938/1
› Further Information
Also available at
   Permissions and Reprints

Referred to by:

Defining the next steps for artificial intelligence in colonoscopyEndoscopy 2021; 53(09): 902-904
DOI: 10.1055/a-1384-0485

Abstract

Background Artificial intelligence (AI) research in colonoscopy is progressing rapidly but widespread clinical implementation is not yet a reality. We aimed to identify the top implementation research priorities.

Methods An established modified Delphi approach for research priority setting was used. Fifteen international experts, including endoscopists and translational computer scientists/engineers, from nine countries participated in an online survey over 9 months. Questions related to AI implementation in colonoscopy were generated as a long-list in the first round, and then scored in two subsequent rounds to identify the top 10 research questions.

Results The top 10 ranked questions were categorized into five themes. Theme 1: clinical trial design/end points (4 questions), related to optimum trial designs for polyp detection and characterization, determining the optimal end points for evaluation of AI, and demonstrating impact on interval cancer rates. Theme 2: technological developments (3 questions), including improving detection of more challenging and advanced lesions, reduction of false-positive rates, and minimizing latency. Theme 3: clinical adoption/integration (1 question), concerning the effective combination of detection and characterization into one workflow. Theme 4: data access/annotation (1 question), concerning more efficient or automated data annotation methods to reduce the burden on human experts. Theme 5: regulatory approval (1 question), related to making regulatory approval processes more efficient.

Conclusions This is the first reported international research priority setting exercise for AI in colonoscopy. The study findings should be used as a framework to guide future research with key stakeholders to accelerate the clinical implementation of AI in endoscopy.

Appendix 1s, 2s, Table 1s



Publication History

Received: 24 June 2020

Accepted: 09 November 2020

Accepted Manuscript online:
09 November 2020

Article published online:
13 January 2021

© 2020. © 2020. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Kudo S, Mori Y, Misawa M. et al. Artificial intelligence and colonoscopy: Current status and future perspectives. Dig Endosc 2019; 31: 363-371
    CrossrefPubMedSearch in Google Scholar
  • 2 Ahmad OF, Soares AS, Mazomenos E. et al. Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions. Lancet Gastroenterol Hepatol 2019; 4: 71-80
    Search in Google Scholar
  • 3 Mori Y, Kudo S, Berzin TM. et al. Computer-aided diagnosis for colonoscopy. Endoscopy 2017; 49: 813-819
    Thieme ConnectPubMedSearch in Google Scholar
  • 4 Wang P, Berzin TM, Glissen Brown JR. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68: 1813-1819
    CrossrefPubMedSearch in Google Scholar
  • 5 Wang P, Liu X, Berzin TM. et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol 2020; 5: 343-351
    Search in Google Scholar
  • 6 Su J-R, Li Z, Shao X-J. et al. Impact of a real-time automatic quality control system on colorectal polyp and adenoma detection: a prospective randomized controlled study (with videos). Gastrointest Endosc 2020; 91: 415-424.e4
    CrossrefPubMedSearch in Google Scholar
  • 7 Liu W-N, Zhang Y-Y, Bian X-Q. et al. Study on detection rate of polyps and adenomas in artificial-intelligence-aided colonoscopy. Saudi J Gastroenterol 2020; 26: 13-19
    CrossrefPubMedSearch in Google Scholar
  • 8 Repici A, Badalamenti M, Maselli R. et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520.e7
    CrossrefPubMedSearch in Google Scholar
  • 9 He J, Baxter SL, Xu J. et al. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019; 25: 30-36
    CrossrefPubMedSearch in Google Scholar
  • 10 Langlotz CP, Allen B, Erickson BJ. et al. A roadmap for foundational research on artificial intelligence in medical imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291: 781-791
    CrossrefPubMedSearch in Google Scholar
  • 11 Vinsard DG, Mori Y, Misawa M. et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest Endosc 2019; 90: 55-63
    CrossrefPubMedSearch in Google Scholar
  • 12 Rees CJ, Ngu WS, Regula J. et al. European Society of Gastrointestinal Endoscopy – Establishing the key unanswered research questions within gastrointestinal endoscopy. Endoscopy 2016; 48: 884-891
    Thieme ConnectPubMedSearch in Google Scholar
  • 13 Francis N, Kazaryan AM, Pietrabissa A. et al. A research agenda for the European Association for Endoscopic Surgeons (EAES). Surg Endosc 2017; 31: 2042-2049
    CrossrefPubMedSearch in Google Scholar
  • 14 Burt CG, Cima RR, Koltun WA. et al. Developing a research agenda for the American Society of Colon and Rectal Surgeons: Results of a Delphi approach. Dis Colon Rectum 2009; 52: 898-905
    CrossrefPubMedSearch in Google Scholar
  • 15 Hart AL, Lomer M, Verjee A. et al. What are the top 10 research questions in the treatment of inflammatory bowel disease? A priority setting partnership with the James Lind Alliance. J Crohns Colitis 2017; 11: 204-211
    CrossrefPubMedSearch in Google Scholar
  • 16 Liu X, Cruz Rivera S, Moher D. et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med 2020; 26: 1364-1374
    CrossrefPubMedSearch in Google Scholar
  • 17 Wang P, Liu P, Glissen Brown JR. et al. Lower adenoma miss rate of computer-aided detection-assisted colonoscopy vs routine white-light colonoscopy in a prospective tandem study. Gastroenterology 2020; 159: 1252-1261.e5
    CrossrefPubMedSearch in Google Scholar
  • 18 Mori Y, Kudo S, Misawa M. et al. Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: A prospective study. Ann Intern Med 2018; 169: 357-366
    CrossrefPubMedSearch in Google Scholar
  • 19 Hassan C, Repici A. The FUSE enigma: Wide-angle or wide-minded?. Gastrointest Endosc 2018; 88: 865-867
    CrossrefPubMedSearch in Google Scholar
  • 20 East JE, Rees CJ. Making optical biopsy a clinical reality in colonoscopy. Lancet Gastroenterol Hepatol 2018; 3: 10-12
    CrossrefPubMedSearch in Google Scholar
  • 21 Bisschops R, East JE, Hassan C. et al. Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline – Update 2019. Endoscopy 2019; 51: 1155-1179
    Thieme ConnectPubMedSearch in Google Scholar
  • 22 Barua I, Vinsard D, Jodal H. et al. Artificial intelligence for polyp detection during colonoscopy: a systematic review and meta-analysis. Endoscopy 2020;
    Thieme ConnectPubMedSearch in Google Scholar
  • 23 Hassan C, Bhandari P, Antonelli G. et al. Artificial intelligence for non-polypoid colorectal neoplasms. Dig Endosc 2020;
    CrossrefPubMedSearch in Google Scholar
  • 24 Bernal J, Tajkbaksh N, Sanchez FJ. et al. Comparative validation of polyp detection methods in video colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge. IEEE Trans Med Imaging 2017; 36: 1231-1249
    CrossrefPubMedSearch in Google Scholar
  • 25 Xu S, Perez M, Yang K. et al. Determination of the latency effects on surgical performance and the acceptable latency levels in telesurgery using the dV-Trainer simulator. Surg Endosc 2014; 28: 2569-2576
    CrossrefPubMedSearch in Google Scholar
  • 26 Byrne MF, Chapados N, Soudan F. et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2019; 68: 94-100
    CrossrefPubMedSearch in Google Scholar
  • 27 Mori Y, Kudo S-E, Misawa M. et al. Simultaneous detection and characterization of diminutive polyps with the use of artificial intelligence during colonoscopy. VideoGIE 2019; 4: 7-10
    CrossrefPubMedSearch in Google Scholar
  • 28 Guizard N, Ghalehjegh SH, Henkel M. et al. Artificial intelligence for real-time multiple polyp detection with identification, tracking, and optical biopsy during colonoscopy. Gastroenterology 2019; 156: S48-S49
    CrossrefPubMedSearch in Google Scholar
  • 29 Sánchez-Montes C, Sánchez FJ, Bernal J. et al. Computer-aided prediction of polyp histology on white light colonoscopy using surface pattern analysis. Endoscopy 2019; 51: 261-265
    Thieme ConnectPubMedSearch in Google Scholar
  • 30 Lutnick B, Ginley B, Govind D. et al. An integrated iterative annotation technique for easing neural network training in medical image analysis. Nat Mach Intell 2019; 1: 112-119
    CrossrefPubMedSearch in Google Scholar
  • 31 Borgli H, Thambawita V, Smedsrud PH. et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data 2020; 7: 283
    CrossrefPubMedSearch in Google Scholar
  • 32 Ahmad OF, Stoyanov D, Lovat LB. Barriers and pitfalls for artificial intelligence in gastroenterology: Ethical and regulatory issues. Tech Innov Gastrointest Endosc 2020; 22: 80-84
    Search in Google Scholar
  • 33 Mori Y, Kudo S, East JE. et al. Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video). Gastrointest Endosc 2020; 92: 905-911.e1
    CrossrefPubMedSearch in Google Scholar