Section 1: Health and Clinical Mangement: The Safety and Quality of Decision Support Systems

Summary

Objectives

The use of clinical decision support systems (CDSS) can improve the overall safety and quality of health care delivery, but may also introduce machine-related errors. Recent concerns about the potential for CDSS to harm patients have generated much debate, but there is little research available to identify the nature of such errors, or quantify their frequency or clinical impact.

Methods

A review of recent literature into electronic prescribing systems, as well as related literature in decision support.

Results

There seems to be some evidence for variation in the outcomes of using CDSS, most likely reflecting variations in clinical setting, culture, training and organizational process, independent of technical variables. There is also preliminary evidence that poorly implemented CDSS can lead to increased mortality in some settings. Studies in the US, UK and Australia have found commercial prescribing systems often fail to uniformly detect significant drug interactions, probably because of errors in their knowledge base. Electronic medication management systems may generate new types of error because of user-interface design, but al so because of events in the workplace such as distraction affecting the actions of system users. Another potential source of CDSS influenced errors are automation biases, including errors of omission where individuals miss important data because the system does not prompt them to notice them, and errors of commission where individuals do what the decision aid tells to do, even when this contradicts their training and other available data. Errors of dismissal occur when relevant alerts are ignored. On-line decision support systems may also result in errors where clinicians come to an incorrect assessment of the evidence, possibly shaped in part by cognitive decision biases.

Conclusions

The effectiveness of decision support systems, like all other health IT, cannot be assessed purely by evaluating the usability and performance of the software, but is the outcome of a complex set of cognitive and socio-technical interactions. A deeper understanding of these issues can result in the design of systems which are not just intrinsically ‘safe’ but which also result in safe outcomes in the hands of busy or poorly resourced clinicians.

Keywords

Decision support systems - safety - error - automation bias - electronic prescribing

• References

• 1 Hunt D, Haynes B, Hanna S, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: A systematic review. J Am Med Assoc 1998; 280 (15) 1339-46.
• 2 Garg A, Adhikari N, McDonald H. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes. A systematic review. J Am Med Assoc 2005; 293 (10) 1223-38.
• 3 Rind D, Safran C, Phillips R, Wang Q, Calkins D, Delbanco T. et al. Effect of computer-based alerts on the treatment and outcomes of hospitalised patients. Arch Intern Med 1994; 154: 1511-7.
• 4 Evans R, Pestotnik S, Classen D, Clemmer T, Weaver L, Orme J. et al. A computer-assisted management program for antibiotics and other anitinfective agents. N Engl J Med 1998; 338 (04) 232-8.
• 5 Protti D. The Application of Computer Technology in GP Surgeries is Beginning To Have Positive Effects on Chronic Disease Management. World View Reports. London: National Health Service; 2005. http://www.connectingforhealth.nhs.uk/worldview/protti6
• 6 Benson T. Why general practitioners use computers and hospital doctors do not: Part 1 incentives. Br Med J 2002; 325: 1086-9.
• 7 Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D. et al. Effect of computerised evidence based guidelines on management of asthma and angina in primary care: cluster randomised controlled trial. Br Med J 2002; 325: 941-6.
• 8 Peters T, Montgomery AA, Fahey T. How accurately do primary health care professionals use cardiovascular risk tables in the management of hypertension?. Br J Gen Pract 1999; 49: 987-8.
• 9 Ash J, Gorman PN, Seshadri V, Hersh W. Computerized physician order entry in U.S. Hospitals: results of a 2002 survey. J Am Med Inform Assoc 2004; 11 (02) 95-9.
• 10 Westbrook J, Gosling AS, Coiera E. Do clinicians use online evidence to support patient care?. A study of 55,000 clinicians. J Am Med Inform Assoc 2004; 11 (02) 113-20.
• 11 Gosling A, Westbrook J, Coiera E. Variation in the use of online clinical evidence: a qualitative analysis. Int J Med Inform 2003; 69: 1-16.
• 12 Ash JS, Gorman PN, Lavelle M, Payne TH, Massaro TA, Frantz GL. et al. A cross-site qualitative study of physician order entry.[see comment]. J Am Med Inform Assoc 2003; 10 (02) 188-200.
• 13 Coiera E. Four rules for the reinvention of health care. BMJ 2004; 328 (7449): 1197-9.
• 14 Wear R, Berg M. Computer Technology and Clinical Work: Still Waiting for Godot. JAMA 2005; 293: 1261-3.
• 15 Nemeth C, Cook RI. Hiding in plain sight: What Koppel et al. tell us about healthcare IT. J Biomed Inform 2005; 38: 262-3.
• 16 Koppel R, Localio A, Cohen A, Strom BL. Neither panacea nor black box: Responding to three Journal of Biomedical Informatics papers on computerized physician order entry system. J Biomed Inform 2005; 38: 267-9.
• 17 Bates DW. Computerized physician order entry and medication errors: finding a balance. J Biomed Inform 2005; 38: 262-3.
• 18 Friedman C, Elstein A, Wolf F, Murphy G, Franz T, Heckerling P. et al. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: A multisite study of 2 systems. JAMA 1999; 282 (19) 1851-6.
• 19 Friedman C, Wyatt J. Evaluation methods in biomedical informatics. Second edition.. New York: Springer; 2005
• 20 Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’Luf N. et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 06 (04) 313-21.
• 21 Graber M. The safety of computer-based medication systems. Arch Intern Med 2004; 164 (03) 339-40.
• 22 Coombes ID, Stowasser DA, Mitchell CA, Varghese P. Effect of computerised prescribing on use of antibiotics. Med J Aust 2004; 180 (03) 140-1.
• 23 Bates DW, Cohen M, Leape L, Overhage JM, Shabot M, Sheridan T. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc 2001; 08 (04) 299-308.
• 24 Ash J, Berg M, Coiera E. Some unintended consequences of information technology in health care: The nature of patient care information systems related errors. JAm Med Inform Assoc 2004; 11 (02) 104-12.
• 25 US Pharmacopiea. MEDMARX: 5th Anniversary Data Report: A Chartbook of 2003 Findings and Trends 1999-2003. 2005
• 26 Koppel R, Metlay J, Cohen A, Abaluck B, Localio A, Kimmel S. et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-203.
• 27 Reason J. Human error. Cambridge: Cambridge University Press; 1990
• 28 Fernando B, Savelyich BS, Avery AJ, Sheikh A, Bainbridge M, Horsfield P. et al. Prescribing safety features of general practice computer systems: evaluation using simulated test cases. BMJ 2004; 328 (7449): 1171-2.
• 29 Liaw S, Kerr S. Computer aided prescribing: decision support needs to be evidence based. Br Med J 2004; 328: 1566.
• 30 Coiera E. Guide to Health Informatics. 2nd Edition ed.. London: Hodder Arnold; 2003
• 31 Skitka L. Does automation bias decision-making?. Int J Hum Comput Stud 1999; 51: 991-1006.
• 32 Wickens C, Hollands J. Engineering Psychology and Human Performance. New Jersey: Prentice Hall; 2000
• 33 Skitka L, Mosier K, Burdick M, Rosenblatt B. Automation bias and errors: are crews better than individuals. Int J Aviat Psychol 2000; 10 (01) 85-97.
• 34 Smith A, Mort M, Goodwin D, Pope C. Making monitoring ‘work’: human-machine interaction and patient safety in anaesthesia. Anaesthesia 2003; 58 (11) 1070.
• 35 Koski E, MkivirtaA SukuvaaraT, Kari A. Frequency and reliability of alarms in the monitoring of cardiac postoperative patients. J Clin Monit Comput 1990; 07 (07) 129-33.
• 36 Lawless S. Crying wolf: false alarms in a pediatric intensive care unit. Crit Care Med 1994; 22 (06) 981-5.
• 37 Galanter WL, Polikaitis A, DiDomenico RJ. A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry. JAm Med Inform Assoc 2004; 11 (04) 270-7.
• 38 Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163 (21) 2625-31.
• 39 Hsieh T, Kuperman GJ, Jaggi T, Hojnowski-Diaz P, Fiskio J, Williams D. et al. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004; 11 (06) 482-91.
• 40 Friedman C, Elstein A, Wolf F, Murphy G, Franz T, Heckerling P. et al. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: A multisite study of 2 systems. JAMA 1999; 282 (19) 1851-6.
• 41 Westbrook J, Coiera E, Gosling AS. Do online information retrieval systems help experienced clinicians answer clinical questions?. J Am Med Inform Assoc 2005; 12 (03) 315-21.
• 42 Lau A, Coiera E. A Bayesian model that predicts the impact of Web searching on decision-making. Journal of American Society of Information Science and Technology. In press 2006
• 43 Open Clinical. Quality and safety of clinical decision support systems a draft protocol for discussion, Open clinical green paper. 2002 http:// www.openclinical.org/docs/int/docs/qands011.pdf
• 44 Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC. et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; Dec; 116 (06) 1506-12.