Download PDF
Thromb Haemost 2010; 104(06): 1258-1262
DOI: 10.1160/TH10-07-0426
New Technologies, Diagnostic Tools and Drugs
Schattauer GmbH

The inter-rater reliability of the Pulmonary Embolism Severity Index

Montserrat Fraga
1   Division of General Internal Medicine University of Lausanne, Lausanne, Switzerland
,
Patrick Taffé
2   Institute of Social and Preventive Medicine, University of Lausanne, Lausanne, Switzerland
,
Marie Méan
1   Division of General Internal Medicine University of Lausanne, Lausanne, Switzerland
,
Olivier Hugli
3   Emergency Department, University of Lausanne, Lausanne, Switzerland
,
Stéphanie Witzig
1   Division of General Internal Medicine University of Lausanne, Lausanne, Switzerland
,
Gérard Waeber
1   Division of General Internal Medicine University of Lausanne, Lausanne, Switzerland
,
Drahomir Aujesky
4   Division of General Internal Medicine, Bern University Hospital, Bern, Switzerland
› Author Affiliations
Further Information

Publication History

Received: 01 July 2010

Accepted after minor revision: 30 July 2010

Publication Date:
24 November 2017 (online)

Also available at
    Permissions and Reprints

Summary

The Pulmonary Embolism Severity Index (PESI) is a validated clinical prognostic model for patients with acute pulmonary embolism (PE). Our goal was to assess the PESI’s inter-rater reliability in patients diagnosed with PE. We prospectively identified consecutive patients diagnosed with PE in the emergency department of a Swiss teaching hospital. For all patients, resident and attending physician raters independently collected the 11 PESI variables. The raters then calculated the PESI total point score and classified patients into one of five PESI risk classes (I-V) and as low (risk classes I/II) versus higher-risk (risk classes III-V). We examined the inter-rater reliability for each of the 11 PESI variables, the PESI total point score, assignment to each of the five PESI risk classes, and classification of patients as low versus higher-risk using kappa (κ) and intra-class correlation coefficients (ICC). Among 48 consecutive patients with an objective diagnosis of PE, reliability coefficients between resident and attending physician raters were > 0.60 for 10 of the 11 variables comprising the PESI. The inter-rater reliability for the PESI total point score (ICC: 0.89, 95% CI: 0.81–0.94), PESI risk class assignment (κ: 0.81, 95% CI: 0.66–0.94), and the classification of patients as low versus higher-risk (κ: 0.92, 95% CI: 0.72–0.98) was near perfect. Our results demonstrate the high reproducibility of the PESI, supporting the use of the PESI for risk stratification of patients with PE.

Keywords

Inter-rater reliability - Pulmonary Embolism Severity Index
 

  • References

  • 1 Aujesky D, Obrosky DS, Stone RA. et al. Derivation and validation of a prognostic model for pulmonary embolism. Am J Respir Crit Care Med 2005; 172: 1041-1046.
    CrossrefPubMedSearch in Google Scholar
  • 2 Aujesky D, Roy PM, Le Manach CP. et al. Validation of a model to predict adverse outcomes in patients with pulmonary embolism. Eur Heart J 2006; 27: 476-481.
    CrossrefPubMedSearch in Google Scholar
  • 3 Jimenez D, Yusen RD, Otero R. et al. Prognostic models for selecting patients with acute pulmonary embolism for initial outpatient therapy. Chest 2007; 132: 24-30.
    CrossrefPubMedSearch in Google Scholar
  • 4 Donze J, Le Gal G, Fine MJ. et al. Prospective validation of the Pulmonary Embolism Severity Index. A clinical prognostic model for pulmonary embolism. Thromb Haemost 2008; 100: 943-948.
    Thieme ConnectPubMedSearch in Google Scholar
  • 5 Wicki J, Perrier A, Perneger TV, Bounameaux H, Junod AF. Predicting adverse outcome in patients with acute pulmonary embolism: a risk score. Thromb Haemost 2000; 84: 548-552.
    Thieme ConnectPubMedSearch in Google Scholar
  • 6 Sanchez O, Trinquart L, Caille V. et al. Prognostic Factors for Pulmonary Embolism: The PREP Study, A Prospective Multicenter Cohort Study. Am J Respir Crit Care Med 2009; 181: 168-173.
    PubMedSearch in Google Scholar
  • 7 Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. J Am Med Assoc 1997; 277: 488-494.
    CrossrefPubMedSearch in Google Scholar
  • 8 Cohen JA. Coefficient of agreement for nominal scales. Ed Psychol Meas 1960; 20: 37-46.
    CrossrefPubMedSearch in Google Scholar
  • 9 Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70: 213-220.
    CrossrefPubMedSearch in Google Scholar
  • 10 Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428.
    CrossrefPubMedSearch in Google Scholar
  • 11 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174.
    CrossrefPubMedSearch in Google Scholar
  • 12 Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85: 257-268.
    CrossrefPubMedSearch in Google Scholar
  • 13 Chan CM, Woods C, Shorr AF. The validation and reproducibility of the Pulmonary Embolism Severity Index. J Thromb Haemost 2010; 8: 1509-1514.
    CrossrefPubMedSearch in Google Scholar
  • 14 Nordenholz KE, Naviaux NW, Stegelmeier K. et al. Pulmonary embolism risk assessment screening tools: the interrater reliability of their criteria. Am J Emerg Med 2007; 25: 285-290.
    CrossrefPubMedSearch in Google Scholar
  • 15 Wolf SJ, McCubbin TR, Feldhaus KM. et al. Prospective validation of Wells Criteria in the evaluation of patients with suspected pulmonary embolism. Ann Emerg Med 2004; 44: 503-510.
    CrossrefPubMedSearch in Google Scholar
  • 16 Runyon MS, Webb WB, Jones AE. et al. Comparison of the unstructured clinician estimate of pretest probability for pulmonary embolism to the Canadian score and the Charlotte rule: a prospective observational study. Acad Emerg Med 2005; 12: 587-593.
    CrossrefPubMedSearch in Google Scholar
  • 17 Jiménez D, Aujesky D, Moores L. et al. Simplification of the Pulmonary Embolism Severity Index for prognostication in patients with acute symptomatic pulmonary embolism. Arch Intern Med 2010; 170: 1383-1389.
    CrossrefPubMedSearch in Google Scholar