Download PDF
CC BY-NC-ND 4.0 · Endosc Int Open 2021; 09(03): E388-E394
DOI: 10.1055/a-1352-3437
Original article

Interobserver agreement of the Paris and simplified classifications of superficial colonic lesions: a Western study

Francesco Cocomazzi
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
2   University of Bari, Section of Gastroenterology, Department of Emergency and Organ Transplantation, Bari, Italy
,
Marco Gentile
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Francesco Perri
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Antonio Merla
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Fabrizio Bossa
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Mariano Piazzolla
2   University of Bari, Section of Gastroenterology, Department of Emergency and Organ Transplantation, Bari, Italy
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Antonio Ippolito
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Fulvia Terracciano
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Arcangela Patrizia Giuliani
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Rossella Cubisino
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Antonella Marra
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Sonia Carparelli
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Alessia Mileti
2   University of Bari, Section of Gastroenterology, Department of Emergency and Organ Transplantation, Bari, Italy
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Rosa Paolillo
2   University of Bari, Section of Gastroenterology, Department of Emergency and Organ Transplantation, Bari, Italy
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
,
Andrea Fontana
3   Unit of Biostatistics, Fondazione “Casa Sollievo della Sofferenza”, IRCCS, San Giovanni Rotondo, Foggia, Italy
,
Massimiliano Copetti
3   Unit of Biostatistics, Fondazione “Casa Sollievo della Sofferenza”, IRCCS, San Giovanni Rotondo, Foggia, Italy
,
Alfredo Di Leo
2   University of Bari, Section of Gastroenterology, Department of Emergency and Organ Transplantation, Bari, Italy
,
Angelo Andriulli
1   Fondazione “Casa Sollievo della Sofferenza”, IRCCS, Gastroenterology and Endoscopy Units, San Giovanni Rotondo, Foggia, Italy
› Author Affiliations
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background and study aims The Paris classification of superficial colonic lesions has been widely adopted, but a simplified description that subgroups the shape into pedunculated, sessile/flat and depressed lesions has been proposed recently. The aim of this study was to evaluate the accuracy and inter-rater agreement among 13 Western endoscopists for the two classification systems.

Methods Seventy video clips of superficial colonic lesions were classified according to the two classifications, and their size estimated. The interobserver agreement for each classification was assessed using both Cohen k and AC1 statistics. Accuracy was taken as the concordance between the standard morphology definition and that made by participants. Sensitivity analyses investigated agreement between trainees (T) and staff members (SM), simple or mixed lesions, distinct lesion phenotypes, and for laterally spreading tumors (LSTs).

Results Overall, the interobserver agreement for the Paris classification was substantial (κ = 0.61; AC1 = 0.66), with 79.3 % accuracy. Between SM and T, the values were superimposable. For size estimation, the agreement was 0.48 by the κ-value, and 0.50 by AC1. For single or mixed lesions, κ-values were 0.60 and 0.43, respectively; corresponding AC1 values were 0.68 and 0.57. Evaluating the several different polyp subtypes separately, agreement differed significantly when analyzed by the k-statistics (0.08–0.12) or the AC1 statistics (0.59–0.71). Analyses of LSTs provided a κ-value of 0.50 and an AC1 score of 0.62, with 77.6 % accuracy. The simplified classification outperformed the Paris classification: κ = 0.68, AC1 = 0.82, accuracy = 91.6 %.

Conclusions Agreement is often measured with Cohen’s κ, but we documented higher levels of agreement when analyzed with the AC1 statistic. The level of agreement was substantial for the Paris classification, and almost perfect for the simplified system.



Publication History

Received: 17 August 2020

Accepted: 09 December 2020

Article published online:
19 February 2021

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commecial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Kudo S, Lambert R, Allen JI. et al. Nonpolypoid neoplastic lesions of the colorectal mucosa. Gastrointest Endosc 2008; 68: S3-S47
    CrossrefPubMedGoogle Scholar
  • 2 The Paris endoscopic classification of superficial neoplastic lesions. esophagus, stomach, and colon. November 30 to December 1, 2002. Gastrointest Endosc 2003; 58: S3-S43
    CrossrefPubMedGoogle Scholar
  • 3 Endoscopic Classification Review Group. Update on the Paris classification of superficial neoplastic lesions in the digestive tract. Endoscopy 2005; 37: 570-578
    Article in Thieme ConnectPubMedGoogle Scholar
  • 4 Ferlitsch M, Moss A, Hassan C. et al. Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline. Endoscopy 2017; 49: 270-297
    Article in Thieme ConnectPubMedGoogle Scholar
  • 5 Kaltenbach T, Anderson JC, Burke CA. et al. Endoscopic removal of colorectal lesions – Recommendations by the US multi-society task force on colorectal cancer. Gastrointest Endosc 2020; 91: 486-519
    CrossrefPubMedGoogle Scholar
  • 6 Japanese Society for Cancer of the Colon and Rectum. Japanese classification of colorectal, appendiceal, and anal carcinoma: the 3rd English version. J Anus Rectum Colon 2019; 3: 175-195
    CrossrefPubMedGoogle Scholar
  • 7 Rao AK, Soetikno R, Raju GS. et al. Large sessile serrated polyps can be safely and effectively removed by endoscopic mucosal resection. Clin Gastroenterol Hepatol 2016; 14: 568-574
    CrossrefPubMedGoogle Scholar
  • 8 Van Doorn SC, Hazewinkel Y, East EJ. et al. Polyp Morphology: an interobserver evaluation for the Paris Classification among international experts. Am J Gastroenterol 2015; 110: 180-187
    CrossrefPubMedGoogle Scholar
  • 9 Kim JH, Nam KS, Kwon HJ. et al. Assessment of colon polyp morphology: is education effective?. World J Gastroenterol 2017; 23: 6281-6286
    CrossrefPubMedGoogle Scholar
  • 10 Aziz Aadam A, Wani S, Kahi CH. et al. Physician assessment and management of complex colon polyps: a multicenter video-based survey study. Am J Gastroenterol 2014; 109: 1312-1317
    CrossrefPubMedGoogle Scholar
  • 11 Lee YJ, Kim ES, Park KS. et al. Inter-observer agreement in the endoscopic classification of colorectal laterally spreading tumors: a multicenter study between experts and trainees. Dig Dis Sci 2014; 59: 2550-2556
    CrossrefPubMedGoogle Scholar
  • 12 Bogie RMM, Veldman MHJ, Snijders LARS. et al. Endoscopic subtypes of colorectal laterally spreading tumors (LSTs) and the risk of submucosal invasion: a meta-analysis. Endoscopy 2018; 50: 263-282
    Article in Thieme ConnectPubMedGoogle Scholar
  • 13 Kottner J, Audigé L, Brorson S. et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 2011; 64: 96-106
    CrossrefPubMedGoogle Scholar
  • 14 Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990; 43: 543-549
    CrossrefPubMedGoogle Scholar
  • 15 Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990; 43: 551-558
    CrossrefPubMedGoogle Scholar
  • 16 Di Eugenio B, Glass M. The kappa statistic: a second look. Comput Linguist 2004; 30: 95-101
    CrossrefPubMedGoogle Scholar
  • 17 Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 2008; 61: 29-48
    CrossrefPubMedGoogle Scholar
  • 18 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174
    CrossrefPubMedGoogle Scholar
  • 19 Vleugels JLA, Hazewinkel Y, Dekker E. Morphological classifications of gastrointestinal lesions. Best Pract Res Clin Gastroenterol 2017; 31: 359-367
    CrossrefPubMedGoogle Scholar
  • 20 Matsuda T, Fujii T, Saito Y. et al. Efficacy of the invasive/non-invasive pattern by magnifying chromoendoscopy to estimate the depth of invasion of early colorectal neoplasms. Am J Gastroenterol 2008; 103: 2700-2706
    CrossrefPubMedGoogle Scholar
  • 21 Iwatate M, Ikumoto T, Hattori S. et al. NBI and NBI combined with magnifying colonoscopy. Diagn Ther Endosc 2012; 2012: 173269
    CrossrefPubMedGoogle Scholar
  • 22 Rex DK, Hassan C, Bourke MJ. The colonscopist’s guide to the vocabulary of colorectal neoplasia: histology, morphology, and management. Gastrointest Endosc 2017; 86: 253-263
    CrossrefPubMedGoogle Scholar
  • 23 Bianco MA, Cipolletta L, Rotondano G. et al. Prevalence of nonpolypoid colorectal neoplasia: an Italian multicenter observational study [published correction appears in Endoscopy. 2010 Jul; 42(7): 563]. Endoscopy 2010; 42: 279-285
    PubMedGoogle Scholar
  • 24 Soetikno RM, Kaltenbach T, Rouse RV. et al. Prevalence of nonpolypoid (flat and depressed) colorectal neoplasms in asymptomatic and symptomatic adults. JAMA 2008; 299: 1027-1035
    CrossrefPubMedGoogle Scholar
  • 25 Sanduleanu S, Rondagh EJ, Masclee AA. Development of expertise in the detection and classification of non-polypoid colorectal neoplasia: Experience-based data at an academic GI unit. Gastrointest Endosc Clin N Am 2010; 20: 449-460
    CrossrefPubMedGoogle Scholar
  • 26 Kim BC, Chang HJ, Han KS. et al. Clinicopathological differences of laterally spreading tumors of the colorectum according to gross appearance. Endoscopy 2011; 43: 100-107
    Article in Thieme ConnectPubMedGoogle Scholar
  • 27 Rembacken BJ, Fujii T, Cairns A. et al. Flat and depressed colonic neoplasms: a prospective study of 1000 colonoscopies in the UK. Lancet 2000; 355: 1211-1214
    CrossrefPubMedGoogle Scholar
  • 28 Puig I, Mármol C, Bustamante M. Endoscopic imaging techniques for detecting early colorectal cancer. Curr Opin Gastroenterol 2019; 35: 432-439
    CrossrefPubMedGoogle Scholar
  • 29 Bisschops R, East JE, Hassan C. et al. Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019. Endoscopy 2019; 51: 1155-1179
    Article in Thieme ConnectPubMedGoogle Scholar