Reducing Therapeutic Duplication in Inpatient Medication Orders

Abstract

Background Therapeutic duplication, the presence of multiple agents prescribed for the same indication without clarification for when each should be used, can contribute to serious medical errors. Joint Commission standards require that orders contain clarifying information about when each order should be given. In our system, as needed (PRN) acetaminophen and ibuprofen orders are major contributors to therapeutic duplication.

Objective The objective of this study is to design and evaluate effectiveness of clinical decision support (CDS) to reduce therapeutic duplication with acetaminophen and ibuprofen orders.

Methods This study was done in a pediatric health system with three freestanding hospitals. We iteratively designed and implemented two CDS strategies aimed at reducing the therapeutic duplication with these agents: (1) interruptive alert prompting clinicians for clarifying PRN comments at order entry and (2) addition of discrete “first-line” and “second-line” PRN reasons to orders. Therapeutic duplications were measured by manual review of orders for 30-day periods before and after each intervention and 6 months later.

Results Therapeutic duplications decreased from 1,485 in the 30 days prior to the first alert implementation to 818 in the 30 days after but rose back to 1,208 in the 30 days prior to the second intervention. After discrete reasons were added to the order, therapeutic duplication decreased to 336 in the immediate 30 days and 6 months later remained at 277. Alerts firing rates decreased from 76.0 per 1,000 PRN acetaminophen or ibuprofen orders to 42.9 after the second intervention.

Conclusion Interruptive alerts may reduce therapeutic duplication but are associated with high rates of user frustration and alert fatigue. Leveraging discrete PRN reasons for “first line” and “second line” produced a greater reduction in therapeutic duplication as well as fewer interruptive alerts and less manual entry for providers.

Keywords

clinical decision support - therapeutic duplication - pediatrics

Background and Significance

Therapeutic duplication is the presence of multiple agents prescribed for the same indication without clarification for when each should be used. Therapeutic duplication can contribute to polypharmacy, serious medical errors, and patient harm.[1] [2] Furthermore, Joint Commission standards require that orders contain clarifying information about when each order should be given, if they are to be given concurrently, or if one order is meant to replace the other.[3] Current approaches to addressing therapeutic duplications include resource-intensive manual approaches such as education, audit, and feedback.[1] [2] [4] [5] Also, most organizations do not have automated checking enabled within their electronic health records (EHRs). Even when automated checking within EHR is included, they are not always effective and contribute to alert fatigue.[6] Further such alerts do not check if the medications are ordered for the same reasons. In a preliminary analysis within our health system, the majority of therapeutic duplications were duplicate as needed (PRN) acetaminophen and ibuprofen orders for either fever, headache, or pain. While some of these therapeutic duplications can be mitigated by careful order set design, acetaminophen and ibuprofen are often ordered outside of order sets or with multiple indications from the same order set. Preference of first-line agent may also vary depending on patient-specific factors, provider preference, and trade-offs between effectiveness and side effects.[7] Alerts are ubiquitously used as a form of CDS; however, they are not always effective in reducing therapeutic duplications.[8] [9] Current literature point to ineffectiveness, but we do not know when alerts may fail to achieve outcomes. Further, only a few studies describe and evaluate the effectiveness of alternative approaches. The aim of this study was to design and evaluate clinical decision support (CDS) for reducing therapeutic duplication for PRN acetaminophen and ibuprofen orders.

Methods

Setting

This study was conducted at a large southeastern pediatric health system with three freestanding children's hospitals using a shared instance of Epic Systems EHR. Prior to the intervention, no specific CDS existed to prevent therapeutic duplication; however, duplicate PRN reasons in acetaminophen and ibuprofen orders had been identified as a risk for citation in internal mock Joint Commission surveys. The study population included all PRN acetaminophen and ibuprofen orders in the inpatient setting.

Interventions

Phase 1

We developed an interruptive alert ([Fig. 1]) for clinicians at order entry for acetaminophen and/or ibuprofen with shared PRN reasons unless there was clarifying verbiage in the order's PRN comments. To target the alert appropriately, we searched our system for common PRN comments on concomitant acetaminophen/ibuprofen pairs with duplicate PRN reasons. We then validated with accreditation experts, which verbiage was acceptably clarifying from a regulatory perspective and organized these into a set of regular expressions programmed into the rules within the alert so that the alert would not fire if an acceptable PRN comment was included in the order. We then performed formative usability testing with pediatric residents using scenarios, a think-aloud protocol,[10] and a test EHR interface to iteratively update the language and images in the representation of the alert.[11] [12] [13] This alert was first implemented on March 10, 2021 in all areas served by the inpatient pharmacy, including inpatient, emergency department, intraoperative, and hospital outpatient departments.

Fig. 1 Alert for therapeutic duplication.

Phase 2

To prevent duplicate PRN indications earlier in the workflow (prior to attempting to sign an order), we adjusted discrete PRN options within each order to include the terms “first line” or “second line” for our most common indications (fever, mild pain, and headache). Indications were arranged vertically such that, for example, “fever (first line)” and “fever (second line)” appeared close to each other. [Fig. 2] shows comparison of appearance of acetaminophen order before and after Phase 2 modifications. If providers do not select a predefined PRN reason, a mandatory PRN comment section appears to accommodate providers to include other reasons. The hard stop disappears if a provider selects a discrete option for PRN reason. These changes were implemented on January 13, 2022.

Fig. 2 (Top) Original PRN reasons before Phase 2. (Bottom) Updated PRN reasons after Phase 2.

Data Extraction and Analysis

Process Measures

We evaluated the number of alert firings per month and per PRN order of acetaminophen or ibuprofen in departments where the alert was active. We also reviewed alert comments for “cranky” comments. Cranky comments are feedback comments about an alert that “includes a list of words, phrases, and punctuation that likely to indicate frustration or annoyance with an alert.”[14] We also passively elicited feedback from end users through a link directly in the alert.

Outcome Measures

We queried the EHR database to identify order pairs for acetaminophen/ibuprofen that were active for the same patient during the same time period. Data were extracted 30 days before and after each intervention implementation. We extracted PRN reasons and comments for these orders. Two trained pharmacists reviewed PRN comments to identify which were sufficiently clarifying according to definitions discussed with accreditation experts on a subset of 100 orders. After establishing interrater reliability, one pharmacist (T.E.D.) reviewed the remaining orders and determined which orders constituted therapeutic duplications. Due to limitations in manual review capacity, we only reviewed the periods 30 days before and after each change and did not follow the number of therapeutic duplications each month.

Statistical Analysis

Proportions were compared using χ² tests. Cohen's kappa was calculated to determine interrater reliability.

Results

In the departments served by the inpatient pharmacy, there were 62,147 PRN acetaminophen and 57,095 PRN ibuprofen orders placed in Phase 1 (March 10, 2021 to January 12, 2022) and 51,117 PRN acetaminophen orders and 47,173 PRN ibuprofen orders placed in Phase 2 (January 13, 22 to December 31, 2022).

Alert Response (Process Measures)

In Phase 1, 9,068 alerts fired (907 alerts per month, 76.0 alerts per 1,000 PRN orders), while 4,213 fired in Phase 2 (364 alerts per month, 42.9 alerts per 1,000 PRN orders, p < 0.001). Qualitative feedback from the alert included 22 (14/22 were “already clarified” or variations thereof) as well as direct outreach from several users who stated they had changed PRN reasons and yet the alert continued to fire. Providers indicated that this alert was firing inappropriately contributing to alert burden and fatigue. On further review, these were found to have involved changes that did not meet rules for clarification set in the alert even though on human review they appeared acceptable (false negatives in our regular expressions). Finally, we also noted providers using dummy entry of free-text PRN reasons (e.g., “.”) to avoid alerts.

Therapeutic Duplications (Outcome Measures)

Cohen's kappa for two pharmacist raters reviewing 100 potential duplicate PRN clarifying verbiage was 1. The total number of orders reviewed with duplicate PRN reasons, orders with clarifying verbiage, and therapeutic duplications for acetaminophen and ibuprofen order pairs are shown in [Fig. 3]. After Phase 1 implementation of interruptive alerts, the absolute number of therapeutic duplications in a 30-day period decreased by 45% from 1,485 to 818, partially due to a 30% reduction in the total number of orders with duplicate PRN reasons as well as a 13.5% improvement in the proportion of such orders with clarifying comments. After Phase 2 implementation, the total number of therapeutic duplications in a 30-day period fell drastically by 73% from 1,208 to 336, primarily due to an 84% decrease in the total number of orders with duplicate PRN reasons, the target of the Phase 2 intervention. These frequencies were grossly unchanged 6 months after Phase 2 implementation.

Fig. 3 Number of therapeutic duplications between acetaminophen and ibuprofen before and after each clinical decision support implementation.

Discussion

Implementation of discrete first-line/second-line PRN reasons produced a sustained reduction in the number of therapeutic duplications between ibuprofen and acetaminophen order pairs. Interruptive CDS alerts targeted at reducing therapeutic duplication between acetaminophen and ibuprofen order pairs produced a modest reduction in the number and percentage of therapeutic duplications. We found that the interruptive alert was associated with considerable provider frustration and alert burden. This mode of CDS also produced a potential reversion to the mean in the period after go live. This effect could be partially attributable to alert fatigue and providers ignoring alert. By contrast, the second intervention focused on error prevention at the time of order entry rather than error correction after a duplication had occurred.[15] [16] The relative success of the second intervention could be attributable to appropriate timely targeting of the decision support during the workflow. When designing alerts, it might be helpful to think about decision support opportunities in other formats and more upstream in the clinician workflow such as in the order sets or documentation templates to help reduce alert burden. In addition to the benefits in therapeutic duplication, this also led to a 44% reduction in alerts per 1,000 PRN orders of acetaminophen or ibuprofen. We were able to identify the burden from the alert in Phase 1 through qualitative feedback that was provided. While 22 comments may look small relative to thousands of alert firings, such comments appear only when there is a problem and are rarely entered when alerts work or even when providers are frustrated that alerts do not work but do not feel agency to improve the system. Informatics teams within organizations should use such “cranky comments” to identify and improve burdensome alerts.[14]

Based on these results, we believe this study demonstrates three major concepts: (1) This study addresses a gap in the literature on approaches to reduce therapeutic duplications. Prior work focused on reducing therapeutic duplications have limited interventions such as education or audit and feedback, which are both resource intensive and require continual maintenance.[1] [2] [4] [5] This CDS-based intervention is more easily scaled and can help reduce therapeutic duplications with minimal ongoing resources for continuous improvement. (2) This study demonstrates the utility of discrete PRN comments as an alternative approach to alerts. This approach can be adopted for other CDS efforts such as dosing adjustments or age-based contraindications where inline notification can provide decision support at the right time in provider workflow and enable better targeting of CDS. (3) The study also shows the importance of evaluation of CDS after implementation to identify effectiveness and unintended effects such as alert burden.

There are several limitations of the current study. First, this is a single-center study at a tertiary care pediatric health system with an enterprise EHR in inpatient setting, which may not generalize to other settings. Second, only inpatient pediatric acetaminophen and ibuprofen therapeutic duplications were evaluated. The potential impact of therapeutic duplication on other medications such as opioids and sedatives is much larger. Future work can adopt inline discrete PRN reasons to avoid duplications and reduce potential adverse events in high-risk medications. Third, we only evaluated PRN medications, there was no evaluation of therapeutic duplications between scheduled medications. Fourth, we had no balancing measures to detect, for example, if our design changes led to differences in how pain was managed. Finally, we only manually evaluated 1 month of data pre- and postimplementation of each intervention. We compensated for this last limitation by manually reviewing PRN indications again 6 months after Phase 2 implementation and did not find evidence of regression to prior error rates.

Conclusion

Discrete PRN reasons specifying “first line” and “second line” dramatically reduced the total number of therapeutic duplications as well as subsequent alerts. This strategy also reduces provider workload by reducing the need to manually enter PRN comments. By contrast, while interruptive alerts may temporarily reduce therapeutic duplications, challenges in directing the user toward the appropriate action can potentiate user frustration and alert fatigue. Future research is needed to evaluate the effectiveness of discrete indications in other PRN and scheduled medications as well as in a broader patient population.

Clinical Relevance Statement

Therapeutic duplications are a regulatory concern and can contribute to serious medical errors. Our work demonstrates an approach that significantly reduced therapeutic duplications in one medication pair at our organization. This work can be extrapolated to other medications and settings.

Multiple-Choice Questions

1. Therapeutic duplication is defined as:

   • The presence of multiple indications for a single medication

   • The presence of multiple medications for a single indication, with clear instructions on when each medication should be administered

   • The presence of multiple medications for a single indication, with no clear instructions on when each medication should be administered

   • The presence of multiple medications each for multiple indications, with clear instructions on when each medication should be administered

   Correct Answer: The correct answer is option c.

2. Which of the following is the best approach to evaluating CDS?

   • Focusing on qualitative feedback from users of the decision support

   • Heuristic review of the decision support artifacts to ensure alignment with design principles

   • Triangulating using measures of decision support burden, user feedback, and impact on clinical outcomes

   • Only randomized trials can be used to evaluate CDS

   Correct Answer: The correct answer is option c.

Conflict of Interest

E.W.O. is a cofounder and has equity in Phrase Health. He is also the Principal Investigator on an R42 grant from the National Library of Medicine (NLM) and National Center for Advancing Translational Science (NCATS). He receives salary support from the NLM and NCATS but no direct revenue from Phrase Health. Other authors have nothing to disclose.

Protection of Human and Animal Subjects

This study was part of Quality Improvement Initiative and was deemed nonhuman subjects by the Children's Healthcare of Atlanta, Institutional Review Board.

Note

This work was presented as a podium abstract at the American Medical Informatics Association Annual Symposium 2022.

• References

• 1 Huynh I, Rajendran T. Therapeutic duplication on the general surgical wards. BMJ Open Qual 2021; 10 (03) e001363
• 2 Witry M, Klein D, Alexander B, Franciscus C, Turvey C. Medication list discrepancies and therapeutic duplications among dual use veterans. Fed Pract 2016; 33 (09) 14-20
• 3 Medication administration – therapeutic duplication versus multimodal therapy. Hospital and Hospital Clinics, Medication Management MM, The Joint Commission. Accessed January 31, 2023 at: https://www.jointcommission.org/standards/standard-faqs/hospital-and-hospital-clinics/medication-management-mm/000002339
• 4 Kan W-C, Kuo SC, Chien TW. et al. Therapeutic duplication in Taiwan hospitals for patients with high blood pressure, sugar, and lipids: evaluation with a mobile health mapping tool. JMIR Med Inform 2020; 8 (07) e11627
• 5 Shao S-C, Lai EC-C, Chan Y-Y, Hung M-J, Chen H-Y. Therapeutic duplication of long-acting injectable drugs. J Patient Saf 2018; 14 (03) e74-e75
• 6 Phansalkar S, Wright A, Kuperman GJ. et al. Towards meaningful medication-related clinical decision support: recommendations for an initial implementation. Appl Clin Inform 2011; 2 (01) 50-62
• 7 Kuo N, Su N-Y, Hou S-K, Kang Y-N. Effects of acetaminophen and ibuprofen monotherapy in febrile children: a meta-analysis of randomized controlled trials. Arch Med Sci 2021; 18 (04) 965-981
• 8 Nabovati E, Vakili-Arki H, Taherzadeh Z. et al. Information technology-based interventions to improve drug-drug interaction outcomes: a systematic review on features and effects. J Med Syst 2017; 41 (01) 12
• 9 Co Z, Holmgren AJ, Classen DC. et al. The development and piloting of the ambulatory electronic health record evaluation tool: lessons learned. Appl Clin Inform 2021; 12 (01) 153-163
• 10 Schumacher RM, Lowry SZ, Locke G, Gallagher PD. NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records National Institute of Standards and Technology. US Department of Commerce; 2010: 1-63
• 11 Kandaswamy S, Gill A, Wood S. et al. User-centered design of central venous access device documentation. JAMIA Open 2022; 5 (01) ooac011
• 12 Mrosak J, Kandaswamy S, Stokes C, Roth D, Dave I, Gillespie S, Orenstein E. The influence of integrating clinical practice guideline order bundles into a general admission order set on guideline adoption. JAMIA open 2021; 4 (04) ooab087
• 13 Orenstein EW, Boudreaux J, Rollins M. et al. Formative usability testing reduces severe blood product ordering errors. Appl Clin Inform 2019; 10 (05) 981-990
• 14 Aaron S, McEvoy DS, Ray S, Hickman T-TT, Wright A. Cranky comments: detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inform Assoc 2019; 26 (01) 37-43
• 15 Campbell R, James R. The five “rights” of clinical decision support. J AHIMA 2013; 84 (10) 42-47 , quiz 48
• 16 Nielsen J, Molich R. Heuristic evaluation of user interfaces. Paper presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 249–256. Seattle, WA: Association for Computing Machinery; 1990

Publication History

Received: 07 February 2023

Accepted: 25 April 2023

Accepted Manuscript online:
27 April 2023

Article published online:
19 July 2023