Deep learning and colon capsule endoscopy: automatic detection of blood and colonic mucosal lesions using a convolutional neural network

Miguel Mascarenhas; Tiago Ribeiro; João Afonso; João P.S. Ferreira; Hélder Cardoso; Patrícia Andrade; Marco P.L. Parente; Renato N. Jorge; Miguel Mascarenhas Saraiva; Guilherme Macedo

doi:10.1055/a-1675-1941

Endoscopy International Open, Inhaltsverzeichnis

CC BY-NC-ND 4.0 · Endosc Int Open 2022; 10(02): E171-E177
DOI: 10.1055/a-1675-1941

Original article

Deep learning and colon capsule endoscopy: automatic detection of blood and colonic mucosal lesions using a convolutional neural network

Miguel Mascarenhas

¹Department of Gastroenterology, São João University Hospital, Porto, Portugal

²WGO Gastroenterology and Hepatology Training Center, Porto, Portugal

³Faculty of Medicine of the University of Porto Porto, Portugal

,

Tiago Ribeiro

¹Department of Gastroenterology, São João University Hospital, Porto, Portugal

²WGO Gastroenterology and Hepatology Training Center, Porto, Portugal

,

João Afonso

¹Department of Gastroenterology, São João University Hospital, Porto, Portugal

²WGO Gastroenterology and Hepatology Training Center, Porto, Portugal

,

João P.S. Ferreira

⁴Department of Mechanical Engineering, Faculty of Engineering of the University of Porto, Porto, Portugal

⁵INEGI – Institute of Science and Innovation in Mechanical and Industrial Engineering, Porto, Portugal.

,

Hélder Cardoso

¹Department of Gastroenterology, São João University Hospital, Porto, Portugal

²WGO Gastroenterology and Hepatology Training Center, Porto, Portugal

³Faculty of Medicine of the University of Porto Porto, Portugal

,

Patrícia Andrade

¹Department of Gastroenterology, São João University Hospital, Porto, Portugal

²WGO Gastroenterology and Hepatology Training Center, Porto, Portugal

³Faculty of Medicine of the University of Porto Porto, Portugal

,

Marco P.L. Parente

⁴Department of Mechanical Engineering, Faculty of Engineering of the University of Porto, Porto, Portugal

⁵INEGI – Institute of Science and Innovation in Mechanical and Industrial Engineering, Porto, Portugal.

,

Renato N. Jorge

⁴Department of Mechanical Engineering, Faculty of Engineering of the University of Porto, Porto, Portugal

⁵INEGI – Institute of Science and Innovation in Mechanical and Industrial Engineering, Porto, Portugal.

,

Miguel Mascarenhas Saraiva

⁶ManopH Gastroenterology Clinic, Porto, Portugal

,

Guilherme Macedo

¹Department of Gastroenterology, São João University Hospital, Porto, Portugal

²WGO Gastroenterology and Hepatology Training Center, Porto, Portugal

³Faculty of Medicine of the University of Porto Porto, Portugal

› Institutsangaben

Abstract

Background and study aims Colon capsule endoscopy (CCE) is a minimally invasive alternative to conventional colonoscopy. However, CCE produces long videos, making its analysis time-consuming and prone to errors. Convolutional neural networks (CNN) are artificial intelligence (AI) algorithms with high performance levels in image analysis. We aimed to develop a deep learning model for automatic identification and differentiation of significant colonic mucosal lesions and blood in CCE images.

Patients and methods A retrospective multicenter study including 124 CCE examinations was conducted for development of a CNN model, using a database of CCE images including anonymized images of patients with normal colon mucosa, several mucosal lesions (erosions, ulcers, vascular lesions and protruding lesions) and luminal blood. For CNN development, 9005 images (3,075 normal mucosa, 3,115 blood and 2,815 mucosal lesions) were ultimately extracted. Two image datasets were created and used for CNN training and validation.

Results The mean (standard deviation) sensitivity and specificity of the CNN were 96.3 % (3.9 %) and 98.2 % (1.8 %) Mucosal lesions were detected with a sensitivity of 92.0 % and a specificity of 98.5 %. Blood was detected with a sensitivity and specificity of 97.2 % and 99.9 %, respectively. The algorithm was 99.2 % sensitive and 99.6 % specific in distinguishing blood from mucosal lesions. The CNN processed 65 frames per second.

Conclusions This is the first CNN-based algorithm to accurately detect and distinguish colonic mucosal lesions and luminal blood in CCE images. AI may improve diagnostic and time efficiency of CCE exams, thus facilitating CCE adoption to routine clinical practice.

Volltext

Referenzen

References
1 Eliakim R, Fireman Z, Gralnek IM. et al. Evaluation of the PillCam Colon capsule in the detection of colonic pathology: results of the first multicenter, prospective, comparative study. Endoscopy 2006; 38: 963-970
2 Niikura R, Yasunaga H, Yamada A. et al. Factors predicting adverse events associated with therapeutic colonoscopy for colorectal neoplasia: a retrospective nationwide study in Japan. Gastrointest Endosc 2016; 84: 971-982.e976
3 Eliakim R, Yassin K, Niv Y. et al. Prospective multicenter performance evaluation of the second-generation colon capsule compared with colonoscopy. Endoscopy 2009; 41: 1026-1031
4 Yasaka K, Akai H, Abe O. et al. Deep Learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: A preliminary study. Radiology 2018; 286: 887-896
5 Esteva A, Kuprel B, Novoa RA. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542: 115-118
6 Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology 2017; 124: 962-969
7 Soffer S, Klang E, Shimon O. et al. Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest Endosc 2020; 92: 831-839.e838
8 Aoki T, Yamada A, Aoyama K. et al. Clinical usefulness of a deep learning-based system as the first screening on small-bowel capsule endoscopy reading. Dig Endosc 2020; 32: 585-591
9 Hosoe N, Limpias Kamiya KJL, Hayashi Y. et al. Current status of colon capsule endoscopy. Dig Endosc 2020;
10 Herrerías-Gutiérrez JM, Argüelles-Arias F, Caunedo-Álvarez A. et al. PillCamColon Capsule for the study of colonic pathology in clinical practice. Study of agreement with colonoscopy. Rev Esp Enferm Dig 2011; 103: 69-75
11 Shi HY, Chan FKL, Higashimori A. et al. A prospective study on second-generation colon capsule endoscopy to detect mucosal lesions and disease activity in ulcerative colitis (with video). Gastrointest Endosc 2017; 86: 1139-1146.e1136
12 Spada C, Hassan C, Galmiche JP. et al. Colon capsule endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2012; 44: 527-536
13 Pedregosa F, Varoquaux G, Gramfort A. et al. Scikit-learn: Machine Learning in Python. J Machine Learning Res 2011; 12: 2825-2830
14 Urban G, Tripathi P, Alkayali T. et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078.e1068
15 Wang K, Lu X, Zhou H. et al. Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study. Gut 2019; 68: 729-741
16 Marya NB, Powers PD, Chari ST. et al. Utilisation of artificial intelligence for the development of an EUS-convolutional neural network model trained to enhance the diagnosis of autoimmune pancreatitis. Gut 2020;
17 Aoki T, Yamada A, Aoyama K. et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2019; 89: 357-363.e352
18 Aoki T, Yamada A, Kato Y. et al. Automatic detection of blood content in capsule endoscopy images based on a deep convolutional neural network. J Gastroenterol Hepatol 2020; 35: 1196-1200
19 Leenhardt R, Vasseur P, Li C. et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc 2019; 89: 189-194
20 Klang E, Barash Y, Margalit RY. et al. Deep learning algorithms for automated detection of Crohnʼs disease ulcers by video capsule endoscopy. Gastrointest Endosc 2020; 91: 606-613.e602
21 Wang X, Qian H, Ciaccio EJ. et al. Celiac disease diagnosis from videocapsule endoscopy images with residual learning and deep feature extraction. Comput Methods Programs Biomed 2020; 187: 105236
22 Saito H, Aoki T, Aoyama K. et al. Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2020; 92: 144-151.e141
23 Spada C, Barbaro F, Andrisani G. et al. Colon capsule endoscopy: What we know and what we would like to know. World J Gastroenterol 2014; 20: 16948-16955
24 Cash BD, Fleisher MR, Fern S. et al. Multicentre, prospective, randomised study comparing the diagnostic yield of colon capsule endoscopy versus CT colonography in a screening population (the TOPAZ study). Gut 2020;
25 Groth S, Krause H, Behrendt R. et al. Capsule colonoscopy increases uptake of colorectal cancer screening. BMC Gastroenterol 2012; 12: 80
26 Holleran G, Leen R, O'Morain C. et al. Colon capsule endoscopy as possible filter test for colonoscopy selection in a screening population with positive fecal immunology. Endoscopy 2014; 46: 473-478
27 Watabe H, Nakamura T, Yamada A. et al. Assessment of an electronic learning system for colon capsule endoscopy: a pilot study. J Gastroenterol 2016; 51: 579-585
28 Blanes-Vidal V, Baatrup G, Nadimi ES. Addressing priority challenges in the detection and assessment of colorectal polyps from capsule endoscopy and colonoscopy in colorectal cancer screening using machine learning. Acta Oncol 2019; 58: S29-s36
29 Yamada A, Niikura R, Otani K. et al. Automatic detection of colorectal neoplasia in wireless colon capsule endoscopic images using a deep convolutional neural network. Endoscopy 2020;
30 Sung J, Ho KY, Chiu HM. et al. The use of Pillcam Colon in assessing mucosal inflammation in ulcerative colitis: a multicenter study. Endoscopy 2012; 44: 754-758
31 Hosoe N, Matsuoka K, Naganuma M. et al. Applicability of second-generation colon capsule endoscope to ulcerative colitis: a clinical feasibility study. J Gastroenterol Hepatol 2013; 28: 1174-1179
32 Ye CA, Gao YJ, Ge ZZ. et al. PillCam colon capsule endoscopy versus conventional colonoscopy for the detection of severity and extent of ulcerative colitis. J Dig Dis 2013; 14: 117-124
33 San Juan-Acosta M, Caunedo-Álvarez A, Argüelles-Arias F. et al. Colon capsule endoscopy is a safe and useful tool to assess disease parameters in patients with ulcerative colitis. Eur J Gastroenterol Hepatol 2014; 26: 894-901
34 D’Haens G, Löwenberg M, Samaan MA. et al. Safety and feasibility of using the second-generation pillcam colon capsule to assess active colonic Crohn's disease. Clin Gastroenterol Hepatol 2015; 13: 1480-1486.e1483