Assessment of competence in ERCP

As endoscopists performing ERCP and researchers in assessment of endoscopic procedures we enjoyed reading the paper by Ekkelenkamp et al. concerning the learning curve for ERCP [1]. We totally agree with the authors that basic competency should not be established on the basis of number of procedures performed, but would like to point out some weaknesses in relying on self-assessment by the trainees. A review of studies comparing self-assessment with observed measures of competence found that physicians had a limited ability to accurately self-assess [2], and a recent study regarding colonoscopic polypectomy found a weak correlation between self-assessment and assessors’ scores [3]. Unfortunately, the deficient accuracy of self-assessment makes it unsuitable as a guide to improving performance, and false optimism about one’s actual competence can be dangerous [4].

Ekkelenkamp et al. used the rate of self-reported successful common bile duct cannulation and other binary outcome measures to construct combined learning curves for 15 trainees. These are very informative and underline the difficulty of mastering ERCP. However, the claims that the self-assessment form can be used to provide “real-time feedback” and to “show the point at which objective performance standards are met” must be questioned. A study on the reliability of the adenoma detection rate used mathematical modelling and found that large sample sizes (e. g. 500) were necessary for a reliable assessment [5]. Objective performance measures should indeed be monitored as part of systematic quality control in the endoscopy unit, but the wide confidence intervals of such measures make them unsuited for the assessment of individual trainees during their learning period. This important task should be done by trained raters using specially designed assessment tools with direct observation or based on video recordings to reduce bias [6]. Automatic computerized assessment systems might also play a role in the future [7].

• References

• 1 Ekkelenkamp VE, Koch AD, Rauws EA et al. Competence development in ERCP: the learning curve of novice trainees. Endoscopy 2014; 46: 949-955
• 2 Davis DA, Mazmanian PE, Fordis M et al. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 2006; 296: 1094-1102
• 3 Ansell J, Hurley JJ, Horwood J et al. Can endoscopists accurately self-assess performance during simulated colonoscopic polypectomy? A prospective, cross-sectional study. Am J Surg 2014; 207: 32-38
• 4 Eva KW, Regehr G, Gruppen LD. Blinded by “insight”: self-assessment and its role in performance improvement. In: Hodges B, Lingard L, , eds. The question of competence. 1st. edn. New York: Ithaca, Cornell University Press; 2012: 131-154
• 5 Do A, Weinberg J, Kakkar A et al. Reliability of adenoma detection rate is based on procedural volume. Gastrointest Endosc 2013; 77: 376-380
• 6 Konge L, Vilmann P, Clementsen P et al. Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer. Endoscopy 2012; 44: 928-933
• 7 Svendsen MB, Preisler L, Hillingsoe JG et al. Using motion capture to assess colonoscopy experience level. World J Gastrointest Endosc 2014; 6: 193-199