Evaluating the Endoscopic Reference Score for eosinophilic esophagitis: moderate to substantial intra- and interobserver reliability

Background and study aims: Recently the Endoscopic Reference Score (EREFS) for endoscopic assessment of eosinophilic esophagitis was introduced, with good interobserver agreement for most signs. The EREFS has not yet been evaluated by other investigators and intraobserver agreement has not been assessed. The aim of this study was to further validate the EREFS by assessing interobserver and intraobserver agreement of endoscopic signs in patients with eosinophilic esophagitis.

Patients and methods: High-quality endoscopic images were made of the esophagus of 30 patients with eosinophilic esophagitis (age 36 years, range 23 – 46 years; 5 female), 6 of whom were in remission. At least three depersonalized images per patient were incorporated into a slideshow. Images were scored by four expert and four trainee endoscopists who were blinded to the patients’ conditions. Interobserver agreement was assessed. After 4 weeks, the images were rescored in a different order to assess intraobserver agreement.

Results: Interobserver agreement was substantial for rings (κ 0.70), white exudates (κ 0.63), and crepe paper esophagus (κ 0.62), moderate for furrows (κ 0.49) and strictures (κ 0.54), and slight for edema (κ 0.12). Intraobserver agreement was substantial for rings (median κ 0.64, IQR 0.46 – 0.70), furrows (median κ 0.69, IQR 0.50 – 0.89), and crepe paper esophagus (median κ 0.69, IQR 0.62 – 0.83), moderate for white exudates (median κ 0.58, IQR 0.54 – 0.71) and strictures (median κ 0.54, IQR 0.33 – 0.70), and less than chance for edema (median κ 0.00, IQR 0.00 – 0.29). Inter- and intraobserver agreement was not substantially different between expert and trainee endoscopists.

Conclusions: Using the EREFS, endoscopic signs of eosinophilic esophagitis were scored consistently by expert and trainee endoscopists.

• References

• 1 Liacouras CA, Furuta GT, Hirano I et al. Eosinophilic esophagitis: updated consensus recommendations for children and adults. J Allergy Clin Immunol 2011; 128: 3-20
• 2 van Rhijn BD, Verheij J, Smout AJ et al. Rapidly increasing incidence of eosinophilic esophagitis in a large cohort. Neurogastroenterol Motil 2013; 25: 47-52
• 3 Prasad GA, Talley NJ, Romero Y et al. Prevalence and predictive factors of eosinophilic esophagitis in patients presenting with dysphagia: a prospective study. Am J Gastroenterol 2007; 102: 2627-2632
• 4 Mackenzie SH, Go M, Chadwick B et al. Eosinophilic oesophagitis in patients presenting with dysphagia – a prospective analysis. Aliment Pharmacol Ther 2008; 28: 1140-1146
• 5 Kim HP, Vance RB, Shaheen NJ et al. The prevalence and diagnostic utility of endoscopic features of eosinophilic esophagitis: a meta-analysis. Clin Gastroenterol Hepatol 2012; 10: 988-996
• 6 Peery AF, Cao H, Dominik R et al. Variable reliability of endoscopic findings with white-light and narrow-band imaging for patients with suspected eosinophilic esophagitis. Clin Gastroenterol Hepatol 2011; 9: 475-480
• 7 Dellon ES, Gonsalves N, Hirano I et al. ACG clinical guideline: Evidenced based approach to the diagnosis and management of esophageal eosinophilia and eosinophilic esophagitis (EoE). Am J Gastroenterol 2013; 108: 679-692
• 8 Hirano I, Moy N, Heckman MG et al. Endoscopic assessment of the oesophageal features of eosinophilic oesophagitis: validation of a novel classification and grading system. Gut 2013; 62: 489-495
• 9 Fleiss JL, Nee JCM, Landis J. Large sample variance of kappa in the case of different sets of raters. Psychol Bull 1979; 1979: 974-977
• 10 Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428
• 11 Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990; 43: 543-549
• 12 Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960; 1960: 37-46
• 13 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174
• 14 van Rhijn BD, van Ree R, Versteeg SA et al. Birch pollen sensitization with cross-reactivity to food allergens predominates in adults with eosinophilic esophagitis. Allergy 2013; 68: 1475-1481