Reliability of In-hospital Mortality as a Quality Indicator in Clinical Quality Registries

Summary

Objectives: Errors in the registration or extraction of patient outcome data, such as in-hospital mortality, may lower the reliability of the quality indicator that uses this (partly) incorrect data. Our aim was to measure the reliability of in-hospital mortality registration in the Dutch National Intensive Care Evaluation (NICE) registry.

Methods: We linked data of the NICE registry with an insurance claims database, resulting in a list of discrepancies in in-hospital mortality. Eleven Intensive Care Units (ICUs) were visited where local data sources were investigated to find the true in-hospital mortality status of the discrepancies and to identify the causes of the data errors in the NICE registry. Original and corrected Stand -ardized Mortality Ratios (SMRs) were calculated to determine if conclusions about quality of care changed compared to the national benchmark.

Results: In eleven ICUs, 23,855 records with 460 discrepancies were identified of which 255 discrepancies (1.1% of all linked records) were due to incorrect in-hospital mortality registration in the NICE registry. Two programming errors in computer software of six ICUs caused 78% of errors, the remainder was caused by manual transcription errors and failure to record patient outcomes. For one ICU the performance became concordant with the national benchmark after correction, instead of being better.

Conclusions: The reliability of in-hospital mortality registration in the NICE registry was good. This was reflected by the low number of data errors and by the fact that conclusions about the quality of care were only affected for one ICU due to systematic data errors. We recommend that registries frequently verify the software used in the registration process, and compare mortality data with an external source to assure consistent quality of data.

Keywords

Quality assurance - root cause analysis - hospital mortality - registries - risk adjustment

• References

• 1 de Jonge E, Bosman RJ, van der Voort PH, Korsten HH, Scheffer GJ, De Keizer NF. Intensive care medicine in the Netherlands, 1997-2001. I. Patient population and treatment outcome. Ned Tijdschr Geneeskd 2003; 147 (21) 1013-1017.
• 2 Harrison DA, Brady AR, Rowan K. Case mix, outcome and length of stay for admissions to adult, general critical care units in England, Wales and Northern Ireland: the Intensive Care National Audit & Research Centre Case Mix Programme Database. Crit Care 2004; 8 (02) R99-111.
• 3 Moran JL, Solomon PJ. Mortality and intensive care volume in ventilated patients from 1995 to 2009 in the Australian and New Zealand binational adult patient intensive care database. Crit Care Med 2012; 40 (03) 800-812.
• 4 Sjoberg F, Walther S. Intensive care registries and the evolution of the concept of ‘quality of care’ - reflections from the 10-year anniversary symposium of the Swedish Intensive Care Registry. Acta Anaesthesiol Scand 2012; 56 (09) 1073-1077.
• 5 Loeb JM. The current state of performance measurement in health care. Int J Qual Health Care 2004; 16 (Suppl. 01) i5-i9.
• 6 Juran JM, Godfrey AB. Juran’s Quality Handbook. Fifth edition. New York: McGraw-Hill; 1999.
• 7 ten Asbroek AH, Arah OA, Geelhoed J, Custers T, Delnoij DM, Klazinga NS. Developing a national performance indicator framework for the Dutch health system. Int J Qual Health Care 2004; 16 (Suppl. 01) i65-i71.
• 8 Roos LL, Wajda A. Record linkage strategies. Part I: Estimating information and evaluating approaches. Methods Inf Med 1991; 30 (02) 117-123.
• 9 Arts DG, De Keizer NF, Scheffer GJ. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc 2002; 9 (06) 600-611.
• 10 Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 1993; 270 (24) 2957-2963.
• 11 Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, de JE, Bosman RJ, Peelen L. et al. External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. J Crit Care 2011; 26 (01) 105-108.
• 12 Moreno R, Apolone G. Impact of different customization strategies in the performance of a general severity score. Crit Care Med 1997; 25 (12) 2001-2008.
• 13 Efron B. Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation. Journal of the American Statistical Association 1983. 78 (382)
• 14 Ji J, Sundquist K, Sundquist J, Hemminki K. Comparability of cancer identification among Death Registry, Cancer Registry and Hospital Discharge Registry. Int J Cancer 2012; 131 (09) 2085-2093.
• 15 Koek HL, Kardaun JW, Gevers E, de BA, Reitsma JB, Grobbee DE. et al. Acute myocardial infarction incidence and hospital mortality: routinely collected national data versus linkage of national registers. Eur J Epidemiol 2007; 22 (11) 755-762.
• 16 Wynn A, Wise M, Wright MJ, Rafaat A, Wang YZ, Steeb G. et al. Accuracy of administrative and trauma registry databases. J Trauma 2001; 51 (03) 464-468.
• 17 Datta I, Findlay C, Kortbeek JB, Hameed SM. Evaluation of a regional trauma registry. Can J Surg 2007; 50 (03) 210-213.
• 18 Oberaigner W, Siebert U. Are survival rates for Tyrol published in the Eurocare studies biased?. Acta Oncol 2009; 48 (07) 984-991.
• 19 Olsson C, Eriksson N, Stahle E, Thelin S. The Swedish Heart Surgery Register: data quality for proximal thoracic aortic operations. Scand Cardiovasc J 2006; 40 (06) 348-353.
• 20 Strobl J, Enzer I, Bagust A, Haycox A, Smyth R, Ashby D. et al. Using disease registries for pharmacoepidemiological research: a case study of data from a cystic fibrosis registry. Pharmacoepidemiol Drug Saf 2003; 12 (06) 467-473.
• 21 Buckley FJ, Poston R. Software Quality Assurance
• 22 Sommerville I. Software Engineering. 8th ed.. Harlow; Pearson Education Limited: 2007. Chapter 23: Software Testing 537-565.
• 23 Bridgewater B. Cardiac registers: the adult cardiac surgery register. Heart 2010; 96 (18) 1441-1443.
• 24 Gjerstorff ML. The Danish Cancer Registry. Scand J Public Health 2011; 39 (Suppl. 07) 42-45.
• 25 Ludvigsson JF, Otterblad-Olausson P, Pettersson BU, Ekbom A. The Swedish personal identity number: possibilities and pitfalls in healthcare and medical research. Eur J Epidemiol 2009; 24 (11) 659-667.
• 26 Lynge E, Sandegaard JL, Rebolj M. The Danish National Patient Register. Scand J Public Health 2011; 39 (Suppl. 07) 30-33.