Download PDF
Endoscopy 2012; 44(10): 928-933
DOI: 10.1055/s-0032-1309892
Original article
© Georg Thieme Verlag KG Stuttgart · New York

Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer

L. Konge
1   Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Copenhagen, Denmark
,
P. Vilmann
2   Department of Surgical Gastroenterology, Copenhagen University Hospital Herlev, Denmark
,
P. Clementsen
3   Department of Pulmonology, Gentofte Hospital, University of Copenhagen, Hellerup, Denmark
,
J. T. Annema
4   Leiden University Medical Center, Pulmonology Department, Leiden, The Netherlands
5   Pulmonology Department, Academical Medical Center, Amsterdam, The Netherlands
,
C. Ringsted
1   Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Copenhagen, Denmark
› Author Affiliations
Further Information

Publication History

submitted 22 September 2011

accepted after revision: 10 May 2012

Publication Date:
23 July 2012 (online)

Also available at
    Permissions and Reprints

Background and study aims: Fine-needle aspiration (FNA) guided by endoscopic ultrasonography (EUS) is important in mediastinal staging of non-small cell lung cancer (NSCLC). Training standards and implementation strategies of this technique are currently under discussion. The aim of this study was to explore the reliability and validity of a newly developed EUS Assessment Tool (EUSAT) designed to measure competence in EUS – FNA for mediastinal staging of NSCLC.

Patients and methods: A total of 30 patients with proven or suspected NSCLC underwent EUS – FNA for mediastinal staging by three trainees and three experienced physicians. Their performances were assessed prospectively by three experts in EUS under direct observation and again 2 months later in a blinded fashion using digital video-recordings. Based on the assessments, intra-rater reliability, inter-rater reliability, and construct validity were explored.

Results: The intra-rater reliability was good (Cronbach’s α = 0.80), but comparison of results based on direct observations and blinded video-recordings indicated a significant bias favoring consultants (P = 0.022). Inter-rater reliability was very good (Cronbach’s α = 0.93). However, one rater assessing five procedures or two raters each assessing four procedures were necessary to secure a generalizability coefficient of 0.80. The assessment tool demonstrated construct validity by discriminating between trainees and experienced physicians (P = 0.034).

Conclusions: Competency in mediastinal staging of NSCLC using EUS and EUS – FNA can be assessed in a reliable and valid way using the EUSAT assessment tool. Measuring and defining competency and training requirements could improve EUS quality and benefit patient care.

 

  • References

  • 1 Youlden DR, Cramb SM, Baade PD. The international epidemiology of lung cancer: geographical distribution and secular trends. J Thorac Oncol 2008; 3: 819-831
    CrossrefPubMedSearch in Google Scholar
  • 2 Chansky K, Sculier JP, Crowley JJ et al. The International Association for the Study of Lung Cancer Staging Project: prognostic factors and pathologic TNM stage in surgically managed non-small cell lung cancer. J Thorac Oncol 2009; 4: 792-801
    CrossrefPubMedSearch in Google Scholar
  • 3 Scott WJ, Howington J, Feigenberg S et al. Treatment of non-small cell lung cancer stage I and stage II: ACCP evidence-based clinical practice guidelines (2nd edition). Chest 2007; 132: 234-242
    CrossrefPubMedSearch in Google Scholar
  • 4 Tomaszek SC, Wigle DA. Surgical management of lung cancer. Semin Respir Crit Care Med 2011; 32: 69-77
    Thieme ConnectPubMedSearch in Google Scholar
  • 5 De LP, Lardinois D, Van Schil PE et al. ESTS guidelines for preoperative lymph node staging for non-small cell lung cancer. Eur J Cardiothorac Surg 2007; 32: 1-8
    CrossrefPubMedSearch in Google Scholar
  • 6 Detterbeck FC, Jantz MA, Wallace M et al. Invasive mediastinal staging of lung cancer: ACCP evidence-based clinical practice guidelines (2nd edition). Chest 2007; 132: 202-220
    CrossrefPubMedSearch in Google Scholar
  • 7 Annema JT, van Meerbeeck JP, Rintoul RC et al. Mediastinoscopy vs endosonography for mediastinal nodal staging of lung cancer: a randomized trial. JAMA 2010; 304: 2245-2252
    CrossrefPubMedSearch in Google Scholar
  • 8 Sanders MK, Faigel DO. Training and simulators. In: Hawes RH, Fockens P, eds. Endosonography. 2nd. edn. Philadelphia: Elsevier Saunders; 2011: 22-28
    CrossrefSearch in Google Scholar
  • 9 Eisen GM, Dominitz JA, Faigel DO et al. Guidelines for credentialing and granting privileges for endoscopic ultrasound. Gastrointest Endosc 2001; 54: 811-814
    CrossrefPubMedSearch in Google Scholar
  • 10 Vilmann P, Saftoiu A. Endoscopic ultrasound-guided fine needle aspiration biopsy: equipment and technique. J Gastroenterol Hepatol 2006; 21: 1646-1655
    CrossrefPubMedSearch in Google Scholar
  • 11 Annema JT, Bohoslavsky R, Burgers S et al. Implementation of endoscopic ultrasound for lung cancer staging. Gastrointest Endosc 2010; 71: 64-70
    CrossrefPubMedSearch in Google Scholar
  • 12 Reznick RK, MacRae H. Teaching surgical skills – changes in the wind. N Engl J Med 2006; 355: 2664-2669
    CrossrefPubMedSearch in Google Scholar
  • 13 Rosch T. State of the art lecture: endoscopic ultrasonography: training and competence. Endoscopy 2006; 38: 69-S72
    Thieme ConnectPubMedSearch in Google Scholar
  • 14 Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002; 287: 226-235
    CrossrefPubMedSearch in Google Scholar
  • 15 Reznick R, Regehr G, MacRae H et al. Testing technical skill via an innovative “bench station” examination. Am J Surg 1997; 173: 226-230
    CrossrefPubMedSearch in Google Scholar
  • 16 Brennan RL. Generalizability theory. 1st. edn. New York: Springer; 2001
    CrossrefSearch in Google Scholar
  • 17 Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The dependability of behavioral measurements. 1st. edn. New York: John Wiley and Sons; 1972
    Search in Google Scholar
  • 18 Vassiliou MC, Feldman LS, Fraser SA et al. Evaluating intraoperative laparoscopic skill: direct observation versus blinded videotaped performances. Surg Innov 2007; 14: 211-216
    CrossrefPubMedSearch in Google Scholar
  • 19 Downing SM. Reliability: on the reproducibility of assessment data. Med Educ 2004; 38: 1006-1012
    CrossrefPubMedSearch in Google Scholar
  • 20 McGaghie WC, Butter J, Kaye M. Observational assessment. In: Downing SM, Yudkowsky R, eds. Assessment in health professions education. 1st. edn. New York: Routledge; 2009: 185-215
    Search in Google Scholar
  • 21 Konge L, Larsen KR, Clementsen P et al. Reliable and valid assessment of clinical bronchoscopy performance. Respiration 2012; 83: 53-60
    CrossrefPubMedSearch in Google Scholar
  • 22 Konge L, Arendrup H, von Buchwald C et al. Using performance in multiple simulated scenarios to assess bronchoscopy skills. Respiration 2011; 81: 483-490
    CrossrefPubMedSearch in Google Scholar
  • 23 Davoudi M, Osann K, Colt HG. Validation of two instruments to assess technical bronchoscopic skill using virtual reality simulation. Respiration 2008; 76: 92-101
    CrossrefPubMedSearch in Google Scholar
  • 24 Larsen CR, Grantcharov T, Schouenborg L et al. Objective assessment of surgical competence in gynaecological laparoscopy: development and validation of a procedure-specific rating scale. BJOG 2008; 115: 908-916
    CrossrefPubMedSearch in Google Scholar
  • 25 Ringsted C, Hodges B, Scherpbier A. “The research compass”: an introduction to research in medical education: AMEE Guide No. 56.. Med Teach 2011; 33: 695-709
    CrossrefPubMedSearch in Google Scholar
  • 26 Magill RA. The stages of learning. Motor learning and control. 8th. edn. New York: McGraw-Hill; 2007: 263-289
    Search in Google Scholar
  • 27 Eloubeidi MA, Tamhane A. EUS-guided FNA of solid pancreatic masses: a learning curve with 300 consecutive procedures. Gastrointest Endosc 2005; 61: 700-708
    CrossrefPubMedSearch in Google Scholar
  • 28 Mertz H, Gautam S. The learning curve for EUS-guided FNA of pancreatic cancer. Gastrointest Endosc 2004; 59: 33-37
    CrossrefPubMedSearch in Google Scholar
  • 29 Polkowski M, Larghi A, Weynand B et al. Learning, techniques, and complications of endoscopic ultrasound (EUS)-guided sampling in gastroenterology: European Society of Gastrointestinal Endoscopy (ESGE) Technical Guideline. Endoscopy 2012; 44: 190-206
    Thieme ConnectPubMedSearch in Google Scholar
  • 30 Konge L, Clementsen P, Larsen KR et al. Establishing pass/fail criteria for bronchoscopy performance. Respiration 2012; 83: 140-146
    CrossrefPubMedSearch in Google Scholar