Real-time deep learning-based colorectal polyp localization on clinical video footage achievable with a wide array of hardware configurations

Abstract

Background and study aims Several computer-assisted polyp detection systems have been proposed, but they have various limitations, from utilizing outdated neural network architectures to a requirement for multi-graphics processing unit (GPU) processing, to validating on small or non-robust datasets. To address these problems, we developed a system based on a state-of-the-art convolutional neural network architecture able to detect polyps in real time on a single GPU and tested on both public datasets and full clinical examination recordings.

Methods The study comprised 165 colonoscopy procedure recordings and 2678 still photos gathered retrospectively. The system was trained on 81,962 polyp frames in total and then tested on footage from 42 colonoscopies and CVC-ClinicDB, CVC-ColonDB, Hyper-Kvasir, and ETIS-Larib public datasets. Clinical videos were evaluated for polyp detection and false-positive rates whereas the public datasets were assessed for F1 score. The system was tested for runtime performance on a wide array of hardware.

Results The performance on public datasets varied from an F1 score of 0.727 to 0.942. On full examination videos, it detected 94 % of the polyps found by the endoscopist with a 3 % false-positive rate and identified additional polyps that were missed during initial video assessment. The system’s runtime fits within the real-time constraints on all but one of the hardware configurations.

Conclusions We have created a polyp detection system with a post-processing pipeline that works in real time on a wide array of hardware. The system does not require extensive computational power, which could help broaden the adaptation of new commercially available systems.

Publication History

Received: 01 October 2020

Accepted: 30 December 2020

Article published online:
22 April 2021

• References

• 1 Dahms S, Nowicki A. Epidemiology and results of treatment of colorectal cancer in Poland. Polish J Surgery 2015; 87: 598-605
• 2 Siegel R, DeSantis C, Jemal A. Colorectal cancer statistics, 2014. CA Cancer J Clin 2014; 64: 104-117
• 3 Brenner H, Chang-Claude J, Jansen L. et al. Reduced risk of colorectal cancer up to 10 years after screening, surveillance, or diagnostic colonoscopy. Gastroenterology 2014; 146: 709-717
• 4 Winawer SJ, Zauber AG, Ho MN. et al. Prevention of colorectal cancer by colonoscopic polypectomy. N Engl J Med 1993; 329: 1977-1981
• 5 Săftoiu A, Hassan C, Areia M. et al. Role of gastrointestinal endoscopy in the screening of digestive tract cancers in Europe: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2020; 52: 293-304
• 6 Kaminski MF, Regula J, Kraszewska E. et al. Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010; 362: 1795-1803
• 7 Corley DA, Jensen CD, Marks AR. et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014; 370: 1298-1306
• 8 Buchner AM, Shahid MW, Heckman MG. et al. Trainee participation is associated with increased small adenoma detection. Gastrointest Endosc 2011; 73: 1223-1231
• 9 Wang P, Berzin TM, Brown JRG. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68: 1813-1819
• 10 Yang YJ, Bang CS. Application of artificial intelligence in gastroenterology. World J Gastroenterol 2019; 25: 1666
• 11 Angermann Q, Aymeric H, Olivier R. Active learning for real time detection of polyps in videocolonoscopy. Medical Image Understanding and Analysis Conference 2016; 90: 182-187
• 12 Otani K, Nakada A, Kurose Y. et al. Automatic detection of different types of small-bowel lesions on capsule endoscopy images using a newly developed deep convolutional neural network. Endoscopy 2020; 52: 786-791
• 13 Cho B-J, Bang CS, Park SW. et al. Automated classification of gastric neoplasms in endoscopic images using a convolutional neural network. Endoscopy 2019; 51: 1121-1129
• 14 Pogorelov K, Ostroukhova O, Jeppsson M. et al. Deep learning and hand-crafted feature based approaches for polyp detection in medical videos. IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS) 2018; 381-386
• 15 Wang P, Xiao X, Brown JRG. et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nature Biomed Engineer 2018; 2: 741-748
• 16 Urban G, Tripathi P, Alkayali T. et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078
• 17 Lee JY, Jeong J, Song EM. et al. Real-time detection of colon polyps during colonoscopy using deep learning: systematic validation with four independent datasets. Sci Rep 2020; 10: 1-9
• 18 Guo Z, Nemoto D, Zhu X. et al. A polyp detection algorithm can detect small polyps: An ex vivo reading test compared with endoscopists. Digest Endosc 2020;
• 19 Angermann Q, Bernal J, Sánchez-Montes C. et al. Towards real-time polyp detection in colonoscopy videos: Adapting still frame-based methodologies for video sequences analysis. In: Computer Assisted and Robotic Endoscopy and Clinical Image-Based Procedures. Springer; 2017: 29-41
• 20 Borgli H, Thambawita V, Smedsrud PH. et al. Hyper-kvasir: A comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Dec 2019. [Online]. Available from: osf.io/mkzcq, Accessed 2020 Jun 3
• 21 Bernal J, Sánchez FJ, Fernández-Esparrach G. et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graphics 2015; 43: 99-111
• 22 Bernal J, Sánchez J, Vilarino F. Towards automatic polyp detection with a polyp appearance model. Pattern Recognition 2012; 45: 3166-3182
• 23 Silva J, Histace A, Romain O. et al. Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J Comput Assist Radiology Surgery 2014; 9: 283-293
• 24 Lin T-Y, Goyal P, Girshick R. et al. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision 2017; 2980-2988
• 25 Tan M, Le Q. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:190511946 2019.
• 26 Wilkins AJ. Visual stress. Oxford University Press; 1995
• 27 Yu L, Chen H, Dou Q. et al. Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J Biomed Health Informat 2016; 21: 65-75
• 28 Mohammed A, Yildirim S, Farup I. et al. Y-net: A deep convolutional neural network for polyp detection. arXiv preprint arXiv:180601907 2018.
• 29 Fan D-P, Ji G-P, Zhou T. et al. Pranet: Parallel reverse attention network for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2020: 263-273
• 30 Guo YB, Matuszewski B. Giana polyp segmentation with fully convolutional dilation neural networks. Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications: SCITEPRESS-Science and Technology Publications 2019; 632-641
• 31 Sornapudi S, Meng F, Yi S. Region-based automated localization of colonoscopy and wireless capsule endoscopy polyps. Applied Sci 2019; 9: 2404
• 32 Qadir HA, Balasingham I, Solhusvik J. et al. Improving automatic polyp detection using cnn by exploiting temporal dependency in colonoscopy video. IEEE J Biomed Health Informat 2019; 24: 180-193
• 33 Dijkstra W, Sobiecki A, Bernal J. et al. Towards a single solution for polyp detection, localization and segmentation in colonoscopy images. In: VISIGRAPP (4: VISAPP). 2019: 616-625
• 34 Bernal J, Tajkbaksh N, Sánchez FJ. et al. Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge. IEEE Transact Med Imaging 2017; 36: 1231-1249
• 35 Gurudu SR, Boroff ES, Crowell MD. et al. Impact of feedback on adenoma detection rates: Outcomes of quality improvement program. J Gastroenterol Hepatol 2018; 33: 645-649