Impact of CPOE on Mortality Rates – Contradictory Findings, Important Messages

Summary

Objective: To analyze the seemingly contradictory results of the Han study (Pediatrics 2005) and the Del Beccaro study (Pediatrics 2006), both analyzing the effect of CPOE systems on mortality rates in pediatric intensive care settings.

Methods: Seven CPOE system experts from the United States and Europe comment on these papers.

Results: The two studies are not contradictory, but almost non-comparable due to differences in design and implementation. They demonstrate the range of outcomes that can be obtained from introducing informatics applications in complex health care settings. Implementing informatics applications is a socio-technical activity, which often depends more on the organizational context than on a specific technology. As health informaticians, we must not only learn from failures, but also avoid both uncritical scepticism that may arise from drawing overly general conclusions from one negative trial, as much as uncritical optimism from limited successful ones.

Conclusion: The commentaries emphasize the need to promote systematic studies for assessing the socio-technical factors that influence the introduction of increasingly sophisticated informatics applications within complex organizations. The emergence of evidence-based health informatics will be based both on evaluation guidelines and implementation guidelines, both of which increase the chances of successful implementation. In addition, well-educated health informaticians are needed to manage and guide the implementation processes.

Keywords

Evaluation studies - CPOE - evidence-based health informatics - mortality

• References

• 1 Kohn L, Corrigan J, Donaldson M. editors To err is Human: Building a Safer Health System. Washington: National Academy Press; 2000
• 2 Evans R. et al A Computer-Assisted Management Program for Antibiotics and Other Antiinfective Agents. NEJM 1998; 338 (04) 232-60.
• 3 Bates D. et al The impact of computerized physician order entry on medication error prevention. JAMIA 1999; 6: 313-21.
• 4 Kuperman GJ, Gibson RF. Computer physician order entry: benefits, costs, and issues. Ann Intern Med 2003; 139 (01) 31-9.
• 5 Garg A. et al Effects of computerised clinical decision support systems on practitioner performance and patient outcomes. A systematic review. JAMA 2005; 293: 1223-38.
• 6 Kaushal R. et al Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163: 1409-16.
• 7 Rothschild J. Computerized physician order entry in the critical care and general inpatient setting: a narrative review. J Crit Care 2004; 19 (04) 271-8.
• 8 Han YY. et al Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; 116: 1506-12.
• 9 Gilhooly K. et al Users’ perceptions of a computerised information system in intensive care (ABICUS) on introduction and after 2 months use. Int J Clin Mon Comput 1991; 8: 101-6.
• 10 Del Beccaro MA. et al Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics 2006; 118: 290-5.
• 11 Sittig DF. et al Lessons from “Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system”. Pediatrics 2006; 118: 797-801.
• 12 Ash JS. et al A cross-site qualitative study of physician order entry. JAMIA 2003; 10: 188-200.
• 13 Ash JS. et al A consensus statement on considerations for a successful CPOE implementation. JAMIA 2003; 10: 229-34.
• 14 Ash JS. et al Perceptions of physician order entry: results of a cross-site qualitative study. Methods Inf Med 2003; 42: 313-23.
• 15 Ash JS. et al Implementing computerized physician order entry: the importance of special people. Int J Med Inform 2003; 69: 235-50.
• 16 Ash J, Coiera E, Berg M. Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System- related Errors. JAMIA 2004; 11: 104-12.
• 17 Campbell EM. et al Types of unintended consequences related to computerized provider order entry. JAMIA 2006; 13: 547-56.
• 18 Ash JS. et al. Principles for a successful computerized physician order entry implementation. AMIA Annu Symp Proc 2003; pp 36-40.
• 19 Ash JS. et al Adding insight: a qualitative crosssite study of physician order entry. Int J Med Inform 2005; 74: 623-8.
• 20 Hosmer D, Lemeshow S. Applied logistic regression. New York: John Wiley & Sons; 1989
• 21 Bastien C, Scapin D. Ergonomic Criteria for the Evaluation of Human Computer Interfaces: INRIA Rocquencourt, Technical Report 156. 1993
• 22 Beuscart-Zéphir M. et al Impact of CPOE on doctor- nurse cooperation for the medication ordering and administration process. Int J Med Inform 2005; 74: 629-41.
• 23 Wears RL, Berg M. Computer technology and clinical work: still waiting for Godot. Jama 2005; 293: 1261-3.
• 24 Ammenwerth E. et al Visions and strategies to improve evaluation of health information systems - reflections and lessons based on the HISEVAL workshop in Innsbruck. Int J Med Inf 2004; 73: 479-91.
• 25 Beuscart-Zephir M, Elkin P, Pelayo S. Human Factors Engineering for clinical applications. Int J Med Inform, in press 2006
• 26 Nielsen J. Usability Engineering. New York: Academic Press; 1993
• 27 Preece J. et al Human-Computer Interaction. New York: Addison-Wesley; 1994
• 28 Hix D, Hartson H. Developing user interfaces: ensuring usability through product and process. New York: John Wiley and Sons; 1993
• 29 Coble JM, Karat J, Orland MJ, Kahn MG. Iterative usability testing: ensuring a usable clinical workstation. ProcAMIAAnnu Fall Symp 1997; 744-8.
• 30 Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int J Hum-Comput Studies 1994; 41: 385-97.
• 31 Patel V, Ramoni M. Cognitive models of directional inference in expert medical reasoning. In Feltovich P. et al (eds). Expertise in Context: Human and Machine. Cambridge MA: MIT Press 1997; pp 67-99.
• 32 Weir C. et al Usability testing as evaluation: Development of a tool. In Cimino J. editor Proceedings of the Twentieth Annual Symposium of Computer Applications in Medical Care 1996; p 870
• 33 Kushniruk A. et al Cognitive evaluaiton of the user interface and vocabulary of an outpatient information system. In Cimino J. editor Proceedings of the 20th Annual Symposium of Computer Applications in Medical Care 1996; pp 22-6.
• 34 Campbell EM. et al. Types of Unintended Consequences Related to Computerized Provider Order Entry. JAMIA 2006
• 35 Shabot M. Ten Commandments for implementing clinical information systems. Proc Bayl Univ Med Cent 2004; 17: 265-9.
• 36 Lorenzi NM, Riley RT. Managing Change: An Overview. JAMIA 2000; 7: 116-24.
• 37 Bonnabry P. et al Use of a prospective risk analysis method to improve the safety of the cancer chemotherapy process. Int J Qual Health Care 2006; 18: 9-16.
• 38 Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study. BMC Med Inform Decis Mak 2006; 6: 3
• 39 Aarts J, Berg M. Same systems, different outcomes - comparing the implementation of computerized physician order entry in two Dutch hospitals. Methods Inf Med 2006; 45: 53-61.
• 40 Koppel R. et al Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293: 1197-2003.
• 41 Brender J. et al Factors influencing success and failure of health informatics systems - a pilot Delphi study. Methods Inf Med 2006; 45: 125-36.
• 42 Talmon J. Evaluation and Implementation: A Call for Action. Methods Inf Med 2006; 45 (Suppl 1) S11-15.
• 43 IMIA. Recommendations of the International Medical Informatics Association (IMIA) on Education in Health and Medical Informatics. Methods Inf Med 2000; 39: 267-77.
• 44 Jaspers MW. et al The International Partnership for Health Informatics Education: lessons learned from six years of experience. Methods Inf Med 2005; 44: 25-31.
• 45 Machan C. et al Publication Bias in Medical Informatics evaluation research: Is it an issue or not? In: Hasman A, et al. (eds). Ubiquity: Technologies for Better Health in Aging Societies. Proceedings of MIE2006. Studies in Health Technology and Informatics vol. 124. Amsterdam: IOS Press; 2006. pp 957-62.
• 46 Ammenwerth E, Shaw N. Bad health informatics can kill - is evaluation the answer?. Methods Inf Med 2005; 44: 1-3.
• 47 Lindberg DA. Research opportunities and challenges in 2005. Methods Inf Med 2005; 44: 483-6.