Download PDF
CC BY-NC-ND 4.0 · Endosc Int Open 2021; 09(07): E1136-E1144
DOI: 10.1055/a-1468-3964
Original article

Multi-expert annotation of Crohn’s disease images of the small bowel for automatic detection using a convolutional recurrent attention neural network

Astrid de Maissin
1   CHD La Roche Sur Yon, department of gastroenterology, La Roche Sur Yon, France
,
Remi Vallée
2   Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Mathurin Flamant
3   Clinique Jules Verne, department of gastroenterology, Nantes, France
,
Marie Fondain-Bossiere
4   CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
,
Catherine Le Berre
4   CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
,
Antoine Coutrot
2   Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Nicolas Normand
2   Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Harold Mouchère
2   Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Sandrine Coudol
5   CHU de Nantes, INSERM CIC 1413, Pôle Hospitalo-Universitaire 11: Santé Publique, Clinique des données, Nantes, France
,
Caroline Trang
4   CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
,
Arnaud Bourreille
4   CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
› Author Affiliations
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background and study aims Computer-aided diagnostic tools using deep neural networks are efficient for detection of lesions in endoscopy but require a huge number of images. The impact of the quality of annotation has not been tested yet. Here we describe a multi-expert annotated dataset of images extracted from capsules from Crohn’s disease patients and the impact of the quality of annotations on the accuracy of a recurrent attention neural network.

Methods Images of capsule were annotated by a reader first and then reviewed by three experts in inflammatory bowel disease. Concordance analysis between experts was evaluated by Fleiss’ kappa and all the discordant images were, again, read by all the endoscopists to obtain a consensus annotation. A recurrent attention neural network developed for the study was tested before and after the consensus annotation. Available neural networks (ResNet and VGGNet) were also tested under the same conditions.

Results The final dataset included 3498 images with 2124 non-pathological (60.7 %), 1360 pathological (38.9 %), and 14 (0.4 %) inconclusive. Agreement of the experts was good for distinguishing pathological and non-pathological images with a kappa of 0.79 (P < 0.0001). The accuracy of our classifier and the available neural networks increased after the consensus annotation with a precision of 93.7 %, sensitivity of 93 %, and specificity of 95 %.

Conclusions The accuracy of the neural network increased with improved annotations, suggesting that the number of images needed for the development of these systems could be diminished using a well-designed dataset.

Supplementary material



Publication History

Received: 25 November 2020

Accepted: 04 March 2021

Article published online:
21 June 2021

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Iddan G, Meron G, Glukhovsky A. et al. Wireless capsule endoscopy. Nature 2000; 405: 417-417
    CrossrefPubMedGoogle Scholar
  • 2 Dionisio PM, Gurudu SR, Leighton JA. et al. Capsule endoscopy has a significantly higher diagnostic yield in patients with suspected and established small-bowel crohn’s disease: a meta-analysis. Am J Gastroenterol 2010; 105: 1240-1248
    CrossrefPubMedGoogle Scholar
  • 3 Böcker U, Dinter D, Litterer C. et al. Comparison of magnetic resonance imaging and video capsule enteroscopy in diagnosing small-bowel pathology: Localization-dependent diagnostic yield. Scand J Gastroenterol 2010; 45: 490-500
    CrossrefPubMedGoogle Scholar
  • 4 Jensen MD, Nathan T, Rafaelsen SR. et al. Diagnostic accuracy of capsule endoscopy for small bowel crohn’s disease is superior to that of MR enterography or CT enterography. Clin Gastroenterol Hepatol 2011; 9: 124-129
    CrossrefPubMedGoogle Scholar
  • 5 González-Suárez B, Rodriguez S, Ricart E. et al. Comparison of capsule endoscopy and magnetic resonance enterography for the assessment of small bowel lesions in Crohnʼs disease. Inflamm Bowel Dis 2018; 24: 775-780
    CrossrefPubMedGoogle Scholar
  • 6 Buisson A, Gonzalez F, Poullenot F. et al. Comparative acceptability and perceived clinical utility of monitoring tools: a nationwide survey of patients with inflammatory bowel disease. Inflamm Bowel Dis 2017; 23: 1425-1433
    CrossrefPubMedGoogle Scholar
  • 7 Le Berre C, Sandborn WJ, Aridhi S. et al. Application of artificial intelligence to gastroenterology and hepatology. Gastroenterology 2020; 158: 76-94.e2
    CrossrefPubMedGoogle Scholar
  • 8 Muhammad K, Khan S, Kumar N. et al. Vision-based personalized Wireless Capsule Endoscopy for smart healthcare: Taxonomy, literature review, opportunities and challenges. Future Gen Comp Sys 2020; 113: 266-280
    CrossrefPubMedGoogle Scholar
  • 9 Yuan Y, Meng MQ-H. Deep learning for polyp recognition in wireless capsule endoscopy images. Med Phys 2017; 44: 1379-1389
    CrossrefPubMedGoogle Scholar
  • 10 Leenhardt R, Vasseur P, Li C. et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc 2019; 89: 189-194
    CrossrefPubMedGoogle Scholar
  • 11 Aoki T, Yamada A, Aoyama K. et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2019; 89: 357-363
    CrossrefPubMedGoogle Scholar
  • 12 Fan S, Xu L, Fan Y. et al. Computer-aided detection of small intestinal ulcer and erosion in wireless capsule endoscopy images. Phys Med Biol 2018; 63: 165001
    CrossrefPubMedGoogle Scholar
  • 13 Alaskar H, Hussain A, Al-Aseem N. et al. Application of convolutional neural networks for automated ulcer detection in wireless capsule endoscopy Images. Sensors (Basel) 2019; 19: 1265
    CrossrefPubMedGoogle Scholar
  • 14 Klang E, Barash Y, Margalit RY. et al. Deep learning algorithms for automated detection of Crohn's disease ulcers by video capsule endoscopy. Gastrointest Endosc 2020; 91: 606-613
    CrossrefPubMedGoogle Scholar
  • 15 Leenhardt R, Li C, Le Mouel JP. et al. CAD-CAP: a 25,000-image database serving the development of artificial intelligence for capsule endoscopy. Endosc Int Open 2020; 8: E415-E420
    Article in Thieme ConnectPubMedGoogle Scholar
  • 16 Ding Z, Shi H, Zhang H. et al. Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology 2019; 157: 1044-1054
    CrossrefPubMedGoogle Scholar
  • 17 Wang S, Xing Y, Zhang L. et al. Deep convolutional neural network for ulcer recognition in wireless capsule endoscopy: experimental feasibility and optimization. Comput Math Methods Med 2019; 2019: 7546215
    CrossrefPubMedGoogle Scholar
  • 18 Leenhardt R, Buisson A, Bourreille A. et al. Nomenclature and semantic descriptions of ulcerative and inflammatory lesions seen in Crohnʼs disease in small bowel capsule endoscopy: An international Delphi consensus statement. United European Gastroenterol J 2020; 8: 99-107
    CrossrefPubMedGoogle Scholar
  • 19 Gralnek IM, Defranchis R, Seidman E. et al. Development of a capsule endoscopy scoring index for small bowel mucosal inflammatory change: development of a capsule endoscopy scoring index. Aliment Pharmacol Ther 2007; 27: 146-154
    CrossrefPubMedGoogle Scholar
  • 20 Vallée R, de Maissin A, Coutrot A. Accurate small bowel lesions detection in wireless capsule endoscopy images using deep recurrent attention neural network. IEEE 21st International Workshop on Multimedia Signal Processing (MMSP 2019), Sep 2019, Kuala Lumpur, Malaysia.
    PubMedGoogle Scholar
  • 21 Vallée R, de Maissin A, Coutrot A. CrohnIPI: An endoscopic image database for the evaluation of automatic Crohn's disease lesions recognition algorithms. Proc SPIE 2020; 11317 DOI: 10.1117/12.2543584.
    CrossrefPubMedGoogle Scholar
  • 22 Deng J, Dong W, Socher R. et al. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conf Comput Vis Pattern Recognit Miami, FL: IEEE; 2009. 248-255 DOI: 10.1109/CVPR.2009.5206848
    CrossrefPubMedGoogle Scholar
  • 23 Cho K, van Merrienboer B, Bahdanau D. et al. On the properties of neural machine translation: encoder-decoder approaches. ArXiv14091259 Cs Stat 2014.
    PubMedGoogle Scholar
  • 24 Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971; 76: 378-382
    CrossrefPubMedGoogle Scholar
  • 25 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ICLR – 2015 conference paper. ArXiv: 1409.1556v6.
    PubMedGoogle Scholar
  • 26 Kaiming He, Xiangyu Z, Shaoqing R. et al. Deep residual learning for image recognition. ILSVRC 2015 conference paper. ArXiv: 1512.03385v1.
    PubMedGoogle Scholar