Download PDF
Appl Clin Inform 2019; 10(05): 927-934
DOI: 10.1055/s-0039-3400447
Research Article
Georg Thieme Verlag KG Stuttgart · New York

The Effect of Eliminating Intermediate Severity Drug-Drug Interaction Alerts on Overall Medication Alert Burden and Acceptance Rate

Amy M. Knight
1   Division of Hospital Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Joyce Maygers
2   Department of Care Management, Johns Hopkins Bayview Medical Center, Baltimore, Maryland, United States
,
Kimberly A. Foltz
3   Division of Clinical Informatics, Department of Information Services, Johns Hopkins Bayview Medical Center, Baltimore, Maryland, United States
,
Isha S. John
4   American Pharmacists Association, Washington, District of Columbia, United States
,
Hsin Chieh Yeh
5   Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States
6   Division of General Internal Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Daniel J. Brotman
6   Division of General Internal Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
› Author Affiliations Funding This publication was made possible by the Johns Hopkins Institute for Clinical and Translational Research (ICTR) which is funded in part by Grant Number UL1 TR 001079 from the National Center for Advancing Translational Sciences (NCATS) a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of the Johns Hopkins ICTR, NCATS or NIH.
Further Information

Publication History

12 March 2019

07 October 2019

Publication Date:
04 December 2019 (online)

Also available at
   Permissions and Reprints

Abstract

Objective This study aimed to determine the effects of reducing the number of drug-drug interaction (DDI) alerts in an order entry system.

Methods Retrospective pre–post analysis at an urban medical center of the rates of medication alerts and alert acceptance during a 5-month period before and 5-month period after the threshold for firing DDI alerts was changed from “intermediate” to “severe.” To ensure that we could determine varying response to each alert type, we took an in-depth look at orders generating single alerts.

Results Before the intervention, 241,915 medication orders were placed, of which 25.6% generated one or more medication alerts; 5.3% of the alerts were accepted. During the postintervention period, 245,757 medication orders were placed of which 16.0% generated one or more medication alerts, a 37.5% relative decrease in alert rate (95% confidence interval [CI]: −38.4 to −36.8%), but only a 9.6% absolute decrease (95% CI: −9.4 to −9.9%). 7.4% of orders generating alerts were accepted postintervention, a 39.6% relative increase in acceptance rate (95% CI: 33.2–47.2%), but only a 2.1% absolute increase (95% CI: 1.8–2.4%). When only orders generating a single medication alert were considered, there was a 69.1% relative decrease in the number of orders generating DDI alerts, and an 85.7% relative increase in the acceptance rate (95% CI: 58.6–126.2%), though only a 1.8% absolute increase (95% CI: 1.3–2.3%).

Conclusion Eliminating intermediate severity DDI alerts resulted in a statistically significant decrease in alert burden and increase in the rate of medication alert acceptance, but alert acceptance remained low overall.

Keywords

clinical decision support - medication alerts - high-alert medications - computerized physician order entry - alert fatigue

Authors' Contributions

All listed authors contributed substantially to the study conception and design or analysis and interpretation of data, drafting the article, or revising it critically for important intellectual content, and final approval of the version to be published. No one who fulfills these criteria has been excluded from authorship.


Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Patients, and was reviewed by Johns Hopkins Institutional Review Board.


 

  • References

  • 1 Shortliffe EH. Computer programs to support clinical decision making. JAMA 1987; 258 (01) 61-66
    CrossrefPubMedSearch in Google Scholar
  • 2 Clinical decision support. Available at: https://www.ahrq.gov/cpi/about/otherwebsites/clinical-decision-support/index.html . Accessed August 13, 2019
    PubMed
  • 3 Bright TJ, Wong A, Dhurjati R. , et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
    CrossrefPubMedSearch in Google Scholar
  • 4 Nuckols TK, Smith-Spangler C, Morton SC. , et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56
    CrossrefPubMedSearch in Google Scholar
  • 5 Murphy DR, Reis B, Sittig DF, Singh H. Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis. Am J Med 2012; 125 (02) 209.e1-209.e7
    CrossrefPubMedSearch in Google Scholar
  • 6 Weingart SN, Simchowitz B, Shiman L. , et al. Clinicians' assessments of electronic medication safety alerts in ambulatory care. Arch Intern Med 2009; 169 (17) 1627-1632
    PubMedSearch in Google Scholar
  • 7 Glassman PA, Simon B, Belperio P, Lanto A. Improving recognition of drug interactions: benefits and barriers to using automated drug alerts. Med Care 2002; 40 (12) 1161-1171
    CrossrefPubMedSearch in Google Scholar
  • 8 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; 13 (05) 547-556
    CrossrefPubMedSearch in Google Scholar
  • 9 Ranji SR, Rennke S, Wachter RM. Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review. BMJ Qual Saf 2014; 23 (09) 773-780
    CrossrefPubMedSearch in Google Scholar
  • 10 Kane-Gill SL, O'Connor MF, Rothschild JM. , et al. Technologic distractions (part 1): Summary of approaches to manage alert quantity with intent to reduce alert fatigue and suggestions for alert fatigue metrics. Crit Care Med 2017; 45 (09) 1481-1488
    CrossrefPubMedSearch in Google Scholar
  • 11 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (02) 138-147
    CrossrefPubMedSearch in Google Scholar
  • 12 Isaac T, Weissman JS, Davis RB. , et al. Overrides of medication alerts in ambulatory care. Arch Intern Med 2009; 169 (03) 305-311
    CrossrefPubMedSearch in Google Scholar
  • 13 Nanji KC, Slight SP, Seger DL. , et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 21 (03) 487-491
    CrossrefPubMedSearch in Google Scholar
  • 14 Carspecken CW, Sharek PJ, Longhurst C, Pageler NM. A clinical case of electronic health record drug alert fatigue: consequences for patient outcome. Pediatrics 2013; 131 (06) e1970-e1973
    CrossrefPubMedSearch in Google Scholar
  • 15 Wachter R. The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine's Computer Age. 1st ed. New York, NY: McGraw Hill Education; 2015
    Search in Google Scholar
  • 16 Wong A, Rehr C, Seger DL. , et al. Evaluation of harm associated with high dose-range clinical decision support overrides in the intensive care unit. Drug Saf 2019; 42 (04) 573-579
    CrossrefPubMedSearch in Google Scholar
  • 17 Knight AM, Falade O, Maygers J, Sevransky JE. Factors associated with medication warning acceptance for hospitalized adults. J Hosp Med 2015; 10 (01) 19-25
    CrossrefPubMedSearch in Google Scholar
  • 18 Thiemann DR. Chair, Johns Hopkins Hospital Clinical Decision Support Committee. Personal communication, 10/14/2013.
    PubMed
  • 19 McAlduff J. , VP-CMIO, MedStar Health, Columbia, Maryland. Email communication, 11/27–28/13
    PubMed
  • 20 Institute for Safe Medication Practice. High-alert medications in acute care setting. Available at: https://www.ismp.org/recommendations/high-alert-medications-acute-list . Accessed August 5, 2019
    PubMed
  • 21 Payne TH, Hines LE, Chan RC. , et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc 2015; 22 (06) 1243-1250
    CrossrefPubMedSearch in Google Scholar
  • 22 Phansalkar S, van der Sijs H, Tucker AD. , et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (03) 489-493
    CrossrefPubMedSearch in Google Scholar
  • 23 Bates DW. CPOE and clinical decision support in hospitals: getting the benefits: comment on “Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction”. Arch Intern Med 2010; 170 (17) 1583-1584
    PubMedSearch in Google Scholar
  • 24 Phansalkar S, Edworthy J, Hellier E. , et al. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc 2010; 17 (05) 493-501
    CrossrefPubMedSearch in Google Scholar
  • 25 Classen DC, Phansalkar S, Bates DW. Critical drug-drug interactions for use in electronic health records systems with computerized physician order entry: review of leading approaches. J Patient Saf 2011; 7 (02) 61-65
    CrossrefPubMedSearch in Google Scholar
  • 26 van der Sijs H, Aarts J, van Gelder T, Berg M, Vulto A. Turning off frequently overridden drug alerts: limited opportunities for doing it safely. J Am Med Inform Assoc 2008; 15 (04) 439-448
    CrossrefPubMedSearch in Google Scholar
  • 27 van der Sijs H, Kowlesar R, Aarts J, Berg M, Vulto A, van Gelder T. Unintended consequences of reducing QT-alert overload in a computerized physician order entry system. Eur J Clin Pharmacol 2009; 65 (09) 919-925
    CrossrefPubMedSearch in Google Scholar
  • 28 Lee EK, Mejia AF, Senior T, Jose J. Improving patient safety through medical alert management: An automated decision tool to reduce alert fatigue. AMIA Annu Symp Proc 2010; 2010: 417-421
    PubMedSearch in Google Scholar
  • 29 Resetar E, Reichley RM, Noirot LA, Doherty JA, Dunagan WC, Bailey TC. Strategies for reducing nuisance alerts in a dose checking application. AMIA Annu Symp Proc 2005; 624-628
    PubMedSearch in Google Scholar
  • 30 Beccaro MA, Villanueva R, Knudson KM, Harvey EM, Langle JM, Paul W. Decision support alerts for medication ordering in a computerized provider order entry (CPOE) system: A systematic approach to decrease alerts. Appl Clin Inform 2010; 1 (03) 346-362
    Thieme ConnectPubMedSearch in Google Scholar
  • 31 Schreiber R, Gregoire JA, Shaha JE, Shaha SH. Think time: a novel approach to analysis of clinicians' behavior after reduction of drug-drug interaction alerts. Int J Med Inform 2017; 97: 59-67
    CrossrefPubMedSearch in Google Scholar
  • 32 Bryant AD, Fletcher GS, Payne TH. Drug interaction alert override rates in the Meaningful Use era: no evidence of progress. Appl Clin Inform 2014; 5 (03) 802-813
    Thieme ConnectPubMedSearch in Google Scholar
  • 33 Dexheimer JW, Kirkendall ES, Kouril M. , et al. The effects of medication alerts on prescriber response in a pediatric hospital. Appl Clin Inform 2017; 8 (02) 491-501
    Thieme ConnectPubMedSearch in Google Scholar
  • 34 Schiff GD, Hickman TT, Volk LA, Bates DW, Wright A. Computerised prescribing for safer medication ordering: still a work in progress. BMJ Qual Saf 2016; 25 (05) 315-319
    CrossrefPubMedSearch in Google Scholar
  • 35 Wright A, Sittig DF, Ash JS. , et al. Governance for clinical decision support: case studies and recommended practices from leading institutions. J Am Med Inform Assoc 2011; 18 (02) 187-194
    CrossrefPubMedSearch in Google Scholar
  • 36 Kawamanto K, Flynn MC, Kukhareva P. , et al. A pragmatic guide to establishing clinical decision support governance and addressing decision support fatigue: A case study. AMIA Annu Symp Proc 2018; 2018: 624-633
    PubMedSearch in Google Scholar
  • 37 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
    Thieme ConnectPubMedSearch in Google Scholar
  • 38 Ko Y, Abarca J, Malone DC. , et al. Practitioners' views on computerized drug-drug interaction alerts in the VA system. J Am Med Inform Assoc 2007; 14 (01) 56-64
    CrossrefPubMedSearch in Google Scholar
  • 39 Spina JR, Glassman PA, Simon B. , et al. Potential safety gaps in order entry and automated drug alerts: a nationwide survey of VA physician self-reported practices with computerized order entry. Med Care 2011; 49 (10) 904-910
    CrossrefPubMedSearch in Google Scholar
  • 40 Beeler PE, Orav EJ, Seger DL, Dykes PC, Bates DW. Provider variation in responses to warnings: do the same providers run stop signs repeatedly?. J Am Med Inform Assoc 2016; 23 (e1): e93-e98
    CrossrefPubMedSearch in Google Scholar
  • 41 Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011; 30 (12) 2310-2317
    CrossrefPubMedSearch in Google Scholar
  • 42 Phansalkar S, Desai AA, Bell D. , et al. High-priority drug-drug interactions for use in electronic health records. J Am Med Inform Assoc 2012; 19 (05) 735-743
    CrossrefPubMedSearch in Google Scholar
  • 43 McEvoy DS, Sittig DF, Hickman TT. , et al. Variation in high-priority drug-drug interaction alerts across institutions and electronic health records. J Am Med Inform Assoc 2017; 24 (02) 331-338
    CrossrefPubMedSearch in Google Scholar
  • 44 Tilson H, Hines LE, McEvoy G. , et al. Recommendations for selecting drug-drug interactions for clinical decision support. Am J Health Syst Pharm 2016; 73 (08) 576-585
    CrossrefPubMedSearch in Google Scholar
  • 45 Nanji KC, Seger DL, Slight SP. , et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc 2018; 25 (05) 476-481
    CrossrefPubMedSearch in Google Scholar
  • 46 Wong A, Amato MG, Seger DL. , et al. Prospective evaluation of medication-related clinical decision support over-rides in the intensive care unit. BMJ Qual Saf 2018; 27 (09) 718-724
    CrossrefPubMedSearch in Google Scholar
  • 47 Phansalkar S, Zachariah M, Seidling HM, Mendes C, Volk L, Bates DW. Evaluation of medication alerts in electronic health records for compliance with human factors principles. J Am Med Inform Assoc 2014; 21 (e2): e332-e340
    CrossrefPubMedSearch in Google Scholar
  • 48 Wong A, Amato MG, Seger DL. , et al. Evaluation of medication-related clinical decision support alert overrides in the intensive care unit. J Crit Care 2017; 39: 156-161
    CrossrefPubMedSearch in Google Scholar
  • 49 Coleman JJ, van der Sijs H, Haefeli WE. , et al. On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. BMC Med Inform Decis Mak 2013; 13: 111
    CrossrefPubMedSearch in Google Scholar
  • 50 Paterno MD, Maviglia SM, Gorman PN. , et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009; 16 (01) 40-46
    CrossrefPubMedSearch in Google Scholar
  • 51 Shah NR, Seger AC, Seger DL. , et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006; 13 (01) 5-11
    CrossrefPubMedSearch in Google Scholar