The Impact of Changes to an Electronic Admission Order Set on Prescribing and Clinical Outcomes in the Intensive Care Unit

Abstract

Background Implementation of disease-specific order sets has improved compliance with standards of care for a variety of diseases. Evidence of the impact admission order sets can have on care is limited.

Objective The main purpose of this article is to evaluate the impact of changes made to an electronic critical care admission order set on provider prescribing patterns and clinical outcomes.

Methods A retrospective, observational before-and-after exploratory study was performed on adult patients admitted to the medical intensive care unit using the Inpatient Critical Care Admission Order Set. The primary outcome measure was the percentage change in the number of orders for scheduled acetaminophen, a histamine-2 receptor antagonist (H2RA), and lactated ringers at admission before implementation of the revised order set compared with after implementation. Secondary outcomes assessed clinical impact of changes made to the order set.

Results The addition of a different dosing strategy for a medication already available on the order set (scheduled acetaminophen vs. as needed acetaminophen) had no impact on physician prescribing (0 vs. 0%, p = 1.000). The addition of a new medication class (an H2RA) to the order set significantly increased the number of patients prescribed an H2RA for stress ulcer prophylaxis (0 vs. 20%, p < 0.001). Rearranging the list of maintenance intravenous fluids to make lactated ringers the first fluid option in place of normal saline significantly decreased the number of orders for lactated ringers (17 vs. 4%, p = 0.005). The order set changes had no significant impact on clinical outcomes such as incidence of transaminitis, gastrointestinal bleed, and acute kidney injury.

Conclusion Making changes to an admission order set can impact provider prescribing patterns. The type of change made to the order set, in addition to the specific medication changed, may have an effect on how influential the changes are on prescribing patterns.

Keywords

admission order set - intensive care unit - prescribing patterns - electronic health record

Background and Significance

An order set is a predefined template that assists health care providers in making clinical decisions when entering orders into the electronic health record (EHR) for medications, laboratory tests, imaging, and diet modifications. It is designed to help clinicians treat specific medical conditions or clinical circumstances according to current guidelines and evidence-based medicine.[1] According to the Institute for Safe Medication Practices, when designed properly order sets can (1) enhance communication about best practices between different disciplines, services, and levels of care; (2) transform practice through evidence-based care; (3) decrease variation and unintended omission through standardized formatting and clear presentation of orders; (4) improve workflow by providing directions that are relevant, easy to understand, intuitively organized, and suitable for direct application to current information-management systems; (5) decrease the risk for medication errors by incorporating safety alerts and reminders; and (6) decrease the number of calls to physicians regarding order clarification.[2]

Implementation of disease-specific orders sets has been shown to improve compliance with standards of care for the management of asthma exacerbations, acute coronary syndrome, septic shock, community acquired pneumonia, and chronic obstructive pulmonary disease.[3] [4] [5] [6] [7] [8] In a study conducted at a large academic medical center, application of a standardized acute coronary syndrome order set significantly increased the number of patients who received aspirin within the first 24 hours of hospitalization.[4] Similarly, initiation of a septic shock order set in the emergency department of a large academic medical center significantly increased the number of patients receiving adequate fluid resuscitation and an appropriate empiric antibiotic regimen, while also reporting decreased 28-day mortality.[5]

While literature supports the utilization of order sets for the treatment of specific disease states, evidence is limited on the impact of admission order sets on outcomes for hospitalized patients. Admission order sets differ from previously studied disease-specific order sets, in that they incorporate a variety of measures related to general patient care such as intravenous (IV) fluids, prophylactic medications, pain medications, and laboratory tests. In most hospitals, usage of admission order sets is higher than that of disease-specific order sets. This is likely due to admission being a frequent event within large hospitals and being a process that requires many orders to be entered simultaneously.[9]

O'Connor et al studied the use of a paper-based admission order set for general medical patients in a community hospital and found a significant increase in the number of patients prescribed deep vein thrombosis prophylaxis, allied health consultations, a diabetic diet, an insulin sliding scale regimen, and the potassium replacement protocol. Use of this admission order set also increased the documentation of allergies and resuscitation status, while decreasing the number of inappropriately ordered laboratory tests.[10] In critically ill patients, Bourdeux et al studied the impact of removing a hydroxyethyl starch (HES) solution from fluid bolus options on an intensive care unit (ICU) admission order set, as well as adding chlorhexidine mouthwash as a default option to the order set. As anticipated, these changes decreased the number of patients receiving HES and increased the number of intubated patients receiving chlorhexidine.[11]

In our health system, a wide variety of order sets exist within the EHR software for the treatment of individual medical conditions, admission to specific hospital units, and pre- and postoperative care. These order sets are routinely updated as new guidelines and literature are released, and they are reviewed by multidisciplinary committees prior to their implementation. On August 20, 2018, three key changes were made to Inpatient Adult Critical Care Admission Order Set (IP CC Admit Order Set) based on new evidence available in the literature. These changes were reviewed and approved by the multidisciplinary Critical Care Performance Improvement Committee.

The first change to this admission order set was implemented to help decrease the need and use of opioids for pain in critically ill patients. In 2017, the United States Department of Health and Human Services declared the opioid crisis a nationwide public health emergency and launched a “5-point strategy” to help combat the epidemic. The strategy includes expanding data collection and research in pain and addiction to improve pain management and addiction treatment.[12] The Society of Critical Care Medicine (SCCM)'s “Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients in the ICU” recommends the use of a multimodal pain management approach in critically ill patients to help decrease opioid consumption and improve analgesic effectiveness and patient outcomes.[13] Acetaminophen is one of several medications recommended for use in a multimodal pain management regimen by not only SCCM's guidelines but also the CDC's “Guideline for Prescribing Opioids for Chronic Pain” and the “Guideline for the Management of Postoperative Pain” created by the American Pain Society and American Society of Anesthesiologists.[14] [15]

The goal of the second change to the admission order set was to decrease prescribing of proton-pump inhibitors (PPIs). Studies have shown histamine-2 receptor antagonist (H2RAs) and PPIs to be equally efficacious for stress ulcer prophylaxis.[16] However, H2RAs may be associated with a lower incidence of Clostridium difficile infections, hospital acquired pneumonia, and ventilator associated pneumonia compared with PPIs.[17] [18] [19] [20]

The final modification to the order set was made to facilitate prescribing of lactated ringers over normal saline. Two studies published in the New England Journal of Medicine in 2018 showed the use of balanced crystalloid solutions, such as lactated ringers and plasma-lyte A, in both critically ill and noncritically ill patients, resulted in a significantly lower incidence of major adverse kidney events within 30 days.[20] [21] Furthermore, unlike balanced crystalloid solutions, fluid resuscitation with normal saline has been associated with hyperchloremic metabolic acidosis, which can lead to several adverse effects including renal vasoconstriction, reduced gastric perfusion, and increased release of inflammatory markers.[23] [24] [25]

The IP CC Admit Order Set is primarily used by rotating medical residents admitting patients to the medical intensive care unit (MICU) at our health system's largest institution. The MICU is a 14-bed unit that operates under a closed ICU model. Education on the order set changes was not provided to the rotating medical residents due to logistic constraints and to limit confounding variables in our study. Due to limited data available regarding the use of admission order sets for clinical decision support (CDS) and the effect of order set design on prescribing, this study sought to evaluate the impact of changes made to a critical care admission order set on both provider prescribing patterns and clinical outcomes. At the time of implementation, it was hypothesized that the order set design changes would increase prescribing of scheduled acetaminophen, H2RAs, and lactated ringers in line with evidence-based recommendations.

Objective

The primary objective of this study was to compare the prescribing of scheduled acetaminophen, an H2RA for stress ulcer prophylaxis, and lactated ringers versus normal saline before and after changes made to the IP CC Admit Order Set.

Methods

A retrospective, observational before-and-after study was performed using data from the EHR. A convenience sample of data was collected for 3 months before the order set changes were implemented (May 19, 2018 to August 19, 2018) and for 3 months after the order set changes were implemented (August 20, 2018 to November 20, 2018).

The first change made to the order set provided an additional dosing strategy for a medication already available on the order set. Originally, only as needed acetaminophen was included on the order set. After order revisions were implemented, scheduled acetaminophen was also available ([Fig. 1A]). The second change added a new medication class to the order set. An H2RA was added to the order set as another option for stress ulcer prophylaxis. Previously, only PPIs were available from the order set for stress ulcer prophylaxis ([Fig. 1B]). The last change to the order set involved rearranging the list of available maintenance IV fluids. Lactated ringers were made the first fluid option on the list to encourage its prescribing over normal saline, which was previously listed as the first fluid option ([Fig. 1C]).

Fig. 1 Before and after images of the changes made to the Inpatient Adult Critical Care Admission Order Set. (A) Addition of a different dosing strategy: acetaminophen. (B) Addition of a new medication class: histamine-2 receptor antagonist (H2RA). (C) Rearranged order of medications: lactated ringers. IV, intravenous; PPI, proton pump inhibitor.

Data from the first 100 patients admitted to the MICU, who met inclusion and exclusion criteria, were used for each group in the study. Data were primarily collected using EHR-generated reports. Data generated from these reports were validated by the authors through manual chart review. A power calculation was not completed as this was a convenience sample of patients available for review during the study period after implementation of the order set. Patients were included if they were greater than or equal to 18 years of age and if providers utilized the IP CC Admit Order Set for entry of their admission orders. Patients were excluded if it was their second admission to the MICU during the same hospital stay or if they were part of a protected population (pregnant or imprisoned).

The primary outcome measure was the percentage change in the number of orders for scheduled acetaminophen, an H2RA, a PPI, and lactated ringers at ICU admission before implementation of the updated order set compared with after implementation of the updated order set. Secondary outcomes assessed the clinical effects of the changes made to the order set. The secondary outcomes measured to assess the clinical impact of adding scheduled acetaminophen to the IP CC Admit Order Set included mean daily dose of opioids in morphine milligram equivalents (MME) administered during the first 5 days of ICU admission and incidence of elevated liver enzyme levels (defined as aspartate transaminase [AST] or alanine transaminase [ALT] levels greater than three times the upper limit of normal). For the addition of an H2RA to the order set, incidence of gastrointestinal (GI) bleeding, stress ulcers, and Clostridium difficile infection was measured. Lastly, the clinical outcomes related to lactated ringers being listed as the first fluid option included incidence of hypernatremia (defined as serum sodium ≥ 150 mmol/L), hyperchloremia (defined as serum chloride ≥ 110 mmol/L), acute kidney injury (AKI) (defined as an increase in serum creatinine [SCr] by ≥ 0.3 mg/dL within 48 hours), continuous venovenous hemofiltration (CVVH), and intermittent dialysis (excluding baseline intermittent dialysis patients).

Minitab 16.1.0 Statistical Software (LEAD Technologies, Inc., Charlotte, North Carolina, United States) was used to complete statistical analysis. Patient demographics, the primary outcome, and secondary outcomes were summarized using descriptive statistics. For continuous data, normality was assessed using the Anderson–Darling test. Significance of variation in normally distributed, continuous data was tested using a two-tailed student t-test, using a predetermined α of less than 0.05. A Mann–Whitney U test was used to analyze variation in continuous data that were not normally distributed. Discrete data were evaluated using a chi-square test or Fisher's exact test.

Results

Three months prior to implementation of order set changes, 314 patients were admitted to the MICU. Of these 314 patients, 259 (82.5%) were admitted using the IP CC Admit Order Set. Three months after order set implementation, 293 (89.3%) out of the 328 patients admitted to the MICU were admitted using the IP CC Admit Order Set. There was no significant difference in baseline patient demographics between the before and after groups ([Table 1]).

Table 1 Baseline patient demographics

Demographic	Before (n = 100)	After (n = 100)	p-Value
Age (year)[a]	62 ± 16	63 ± 17	0.783
Sex (female)[b]	45	42	0.669
Weight (kg)[c]	75 (62.98)	81 (66.97)	0.217
Hospital LOS (days)[c]	5 (3.10)	6 (3.12)	0.422
MICU LOS (days)[c]	2 (1.3)	2 (1.4)	0.292
PTA scheduled acetaminophen[c]	0 (0)	3 (3)	0.246
PTA H2RA[b]	8	9	0.800
PTA PPI[b]	25	24	0.869
PTA iHD[b]	3	10	0.082

Abbreviations: H2RA, histamine-2 receptor antagonist; iHD, intermittent hemodialysis; IQR, interquartile range; LOS, length of stay; MICU, medical intensive care unit; PPI, proton pump inhibitor; PTA, prior to admission.

^a Mean ± standard deviation.

^b n.

^c Median (IQR).

There was no significant difference in the number of orders for scheduled acetaminophen before and after implementation of the order set changes (0 vs. 0%, p = 1.000). No providers ordered scheduled acetaminophen from the order set in 3 months after the implementation of order set changes ([Fig. 2]). There was no significant difference in the clinical outcomes of adding scheduled acetaminophen to the order set. The number of patients experiencing elevated liver enzymes was similar in the before and after groups (AST 21 vs. 17%, p = 0.470; ALT 14 vs. 17%, p = 0.692), as were each group's median [interquartile range] opioid requirements during the first 5 days in the ICU (30 [0.96] MME vs. 15 [0.96] MME, p = 0.875) ([Table 2]).

Table 2 Clinical outcomes of changes to IP CC Admit Order Set

Clinical outcome	Before (n = 100)	After (n = 100)	p-Value
Addition of a different dosing strategy: acetaminophen
Opioid requirements, First 5 days in the MICU (MME)[a]	30 (0.96)	15 (0.96)	0.875
AST > 3x ULN (U/L)[b]	21	17	0.470
ALT > 3x ULN (U/L)[b]	14	17	0.692
Addition of a new medication class: histamine-2 receptor antagonist
Gastrointestinal bleed[b]	1	1	1.000
Stress ulcer[b]	0	0	1.000
Clostridium difficile infection[b]	2	1	1.000
Rearranged order of medications: lactated ringers
Hypernatremia[b]	11	11	1.000
Hyperchloremia[b]	25	35	0.121
AKI[b]	13	16	0.546
CVVH[b]	3	6	0.498
iHD[b]	2	1	1.000

Abbreviations: AKI, acute kidney injury; ALT, alanine transaminase; AST, aspartate transaminase; CVVH, continuous venovenous hemofiltration; iHD, intermittent hemodialysis; IP CC Admit Order Set; inpatient adult critical care admission order set; IQR, interquartile range; MME, morphine milligram equivalents.

^a Median (IQR).

^b n.

Fig. 2 Percentage of orders for each medication before and after implementation of order set changes. APAP, acetaminophen; H2RA, histamine-2 receptor antagonist; IV, intravenous; LR, lactated ringers; NS, normal saline; PPI, proton pump inhibitor.

After the addition of an H2RA to the order set, there was a significant increase in the number of patients prescribed an H2RA for stress ulcer prophylaxis (0 vs. 20%, p < 0.001). The increase in H2RA orders in the after group was coupled with a significant decrease in the number of patients prescribed a PPI from the order set (45 vs. 25%, p = 0.004) ([Fig. 2]). Regarding clinical outcomes related to adding an H2RA to the order set, there was no significant difference in the incidence of GI bleed (1 vs. 1%, p = 1.000), stress ulcers (0 vs. 0%, p = 1.000), or Clostridium difficile infections (2 vs. 1%, p = 1.000) between the before and after groups ([Table 2]).

Moving lactated ringers to the first IV fluid option on the order set had no effect on the percentage of orders for other IV fluids (IV fluids besides lactated ringers and normal saline) made from the order set (0 vs. 0%, p = 1.000). There were significantly more orders for lactated ringers in the before group than the after group (17 vs. 4%, p = 0.005). The percentage of orders for normal saline was lower in the after group than in the before group, but not statistically significant (18 vs. 9%, p = 0.06) ([Fig. 2]). Making lactated ringers the first IV fluid option on the order set had no significant impact on the incidence of hypernatremia (11 vs. 11%, p = 1.000), hyperchloremia (25 vs. 35%, p = 0.121), AKI (13 vs. 16%, = 0.546), or need for CVVH (3 vs. 6%, p = 0.498) or intermittent hemodialysis (2 vs. 1%, p = 1.000) ([Table 2]).

Discussion

This before and after study assessed the impact of three different changes to an admission order set on physician prescribing. The addition of a new dosing strategy for a medication already available on the order set had no effect on prescribing. The addition of a new medication class to the order set significantly increased its prescribing and rearranging the order of a list of medications led to a counterintuitive decrease in prescribing of all medications in the list.

Unfortunately, no prescribers ordered scheduled acetaminophen using the IP CC Admit Order Set and no conclusion related to scheduled acetaminophen's impact on patients' opioid requirements was able to be drawn. It is unclear why the addition of scheduled acetaminophen to the order set did not encourage physicians to order it as we did not directly observe order entry, explore practitioners' values surrounding acetaminophen ordering, or ask staff about their opinions regarding acetaminophen use in the MICU. Since acetaminophen had previously been available on the order set, it is possible that providers may have overlooked that acetaminophen was now available with a different dosing frequency (scheduled vs. as needed). Another contributing factor related to the absence of scheduled acetaminophen orders may have been that many patients admitted to the MICU have acute liver impairment or shock liver; thus, inclusion of only MICU patients may have limited the use of scheduled acetaminophen.

While the addition of a new dosing strategy to the order set had no impact on prescribing, the addition of a new medication class did. The addition of an H2RA to the order set significantly increased the number of orders for an H2RA, while significantly decreasing the number of orders for PPIs. These changes in prescribing did not have a significant impact on clinical outcomes, which is likely due to the small sample size.

Surprisingly, there were significantly less orders for lactated ringers in the after group compared with the before group, suggesting that making lactated ringers the first fluid option had the opposite effect intended. This is unexpected as it has been shown that placing high frequency items at the top of a list or menu speeds performance and improves preference ratings.[26] However, the number of normal saline orders also decreased in the after population, although not statistically significant. These results may be impacted by the fact that the after group required less maintenance fluids than the before group. Furthermore, IV fluids are often ordered and administered in the emergency department at our institution. The majority of emergency department order sets contain normal saline as their primary fluid. Thus, providers may have chosen to continue the IV fluid initiated in the emergency department rather than starting a new IV fluid once the patient was admitted and transported to the MICU. Other possible explanations for more use of normal saline than lactated ringers are that normal saline is compatible with more IV medications and more often used to keep a patient's vein open to maintain an IV site. There were no orders for other IV fluids (IV fluids besides lactated ringers and normal saline) made from the order set. Additionally, plasma-lyte A, another balanced crystalloid solution used for fluid resuscitation, is not available on the order set due to cost restraints, but it is anecdotally ordered by many providers outside of the order set. Making lactated ringers the first IV fluid option on the order set had no statistically significant impact on the incidence of hypernatremia, hyperchloremia, AKI, and need for CVVH or intermittent hemodialysis. However, although not statistically significant, with less lactated ringers ordered in the after group, there was an increase in hyperchloremia, AKI, and need for CVVH.

Previous studies on order set design have shown that changing the default settings of items on an order set can significantly influence prescribing behavior. Jacobs et al changed the default setting of three items on a pediatric inpatient asthma care order set from “off” to “on” resulting in a 15% increase in weight measurement orders, an 11% increase in activity center orders, and a 40% increase in peak flow measurement orders.[27] Similarly, Olson et al found that when the default option for posttransfusion laboratories was changed from “optional” to “preselected,” the rate of transfusions with posttransfusion laboratories increased fivefold. This same degree change in the opposite direction was observed when the default order for platelet count was switched back from “preselected” to “optional.”[28] Furthermore, Leis et al showed that simple removal of a check box for a thyroid-stimulating hormone (TSH) order from a critical care unit admission order set significantly decreased inappropriately ordered TSH tests from 60.6 to 20%.[29]

This study is unique in that it sought to assess the effects of several different types of order sets changes outside of modifying the default settings of an order. Order set design is an important area of study as computerized provider order entry (CPOE) with CDS can lead to unintended consequences that need to be monitored for after implementation.[30] [31] Unintended consequences include workflow inefficiency, excessive resource use, reduced confidence in CPOE and CDS systems, and clinical errors.[31] For example, Rosenbloom et al introduced an intervention intended to help decrease unnecessary orders for serum magnesium levels that led to an unanticipated increase in orders for serum magnesium levels.[32] Other strengths of this study are that the order set studied is one used frequently by prescribers, and prescribers were blinded to the time period of data collection, which helped reduce bias.

There are some limitations that should be taken into account when interpreting the results of this study. First, because this is a before and after exploratory study, limited to 6 months of data, the results may be influenced by other unmeasured factors. The diagnoses of patients admitted using the IP CC Admit Order Set were not collected, which may have provided insight as to why fluid orders were lower in the after group than the before group and why scheduled acetaminophen was not ordered by physicians. The retrospective design of our single center study and the use of a convenience sample are also limitations. Thus, our results should be viewed as hypothesis-generating. Future studies could include a comparison of physician prescribing among several ICUs within a single institution; however, this was outside the scope of this study. The same provider may be responsible for the admission of several patients and the ordering tendency of each provider was not evaluated. Additionally, there were no restrictions to providers ordering scheduled acetaminophen, an H2RA, and/or maintenance IV fluids outside of the order set, and these orders were not accounted for in the collection of data. This, however, may represent a more real-world approach, thus increasing the external validity of the study.

Conclusion

Making seemingly minor changes to an order set can have a significant impact on prescribing patterns. Due to their practice altering potential, it is important for health systems to assess for intended and unintended changes in prescribing patterns after implementing or redesigning order sets. The type of modification made to the order set may have an effect on how influential the changes made are on prescribing patterns. Future studies should assess how the type of intervention made to an order set can affect prescribing to help answer the question—is it the medication or the type of intervention made to the order set that affects provider prescribing?

Clinical Relevance Statement

The results of this study support the use of admission order sets to drive provider prescribing patterns. Properly designed order sets based upon guideline recommendations can promote evidence-based prescribing of medications in the ICU.

Multiple Choice Questions

1. Disease specific order sets have been shown to_______________.

   • Improve compliance with standards of care.

   • Complicate physician.

   • Increase the number of inappropriate laboratory tests.

   • Have no effect on patient outcomes.

   Correct Answer: The correct answer is option a, improve compliance with standards of care. Implementation of disease-specific orders sets has been shown to improve compliance with standards of care for the management of asthma exacerbations, acute coronary syndrome, septic shock, community acquired pneumonia, and chronic obstructive pulmonary disease.[3] [4] [5] [6] [7] [8]

2. Which of the following order set changes has significantly influenced prescribing behavior in numerous studies?

   • Changing the default settings of an order.

   • Addition of a new dosing strategy for a medication already available on the order set.

   • Adding a new medication class to the order set.

   • Rearranging the order of how medications are listed in the order set.

   Correct Answer: The correct answer is option a, changing the default settings of an order. Bourdeux et al,[11] Jacobs et al,[27] and Olson et al[28] all found that adjusting the default settings of an order on an order set significantly changed prescribing of an order. In this study, adding a new medication class to an order set had a significant impact on prescribing. However, this had not been studied previously.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

This study was conducted in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was approved by the Institutional Review Board for Western Michigan University Homer Stryker M.D. School of Medicine.

• References

• 1 Cowden D, Barbacioru C, Kahwash E, Saltz J. Order sets utilization in a clinical order entry system. AMIA Annu Symp Proc 2003; 2003: 819
• 2 ISMP's Guidelines for Standard Order Sets. Institute for Safe Medication Practices. https://www.ismp.org/guidelines/standard-order-sets . Published March 11, 2010. Accessed August 15, 2018
• 3 Chisolm DJ, McAlearney AS, Veneris S, Fisher D, Holtzlander M, McCoy KS. The role of computerized order sets in pediatric inpatient asthma treatment. Pediatr Allergy Immunol 2006; 17 (03) 199-206
• 4 Ozdas A, Speroff T, Waitman LR, Ozbolt J, Butler J, Miller RA. Integrating “best of care” protocols into clinicians' workflow via care provider order entry: impact on quality-of-care indicators for acute myocardial infarction. J Am Med Inform Assoc 2006; 13 (02) 188-196
• 5 Micek ST, Roubinian N, Heuring T. , et al. Before-after study of a standardized hospital order set for the management of septic shock. Crit Care Med 2006; 34 (11) 2707-2713
• 6 Fargo EL, D'Amico F, Pickering A, Fowler K, Campbell R, Baumgartner M. Impact of electronic physician order-set on antibiotic ordering time in septic patients in the emergency department. Appl Clin Inform 2018; 9 (04) 869-874
• 7 Krive J, Shoolin JS, Zink SD. Effectiveness of evidence-based pneumonia CPOE order sets measured by health outcomes. Online J Public Health Inform 2015; 7 (02) e211
• 8 Brown KE, Johnson KJ, DeRonne BM, Parenti CM, Rice KL. Order set to improve the care of patients hospitalized for an exacerbation of chronic obstructive pulmonary disease. Ann Am Thorac Soc 2016; 13 (06) 811-815
• 9 Wright A, Feblowitz JC, Pang JE. , et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform 2012; 81 (11) 733-745
• 10 O'Connor C, Adhikari NKJ, DeCaire K, Friedrich JO. Medical admission order sets to improve deep vein thrombosis prophylaxis rates and other outcomes. J Hosp Med 2009; 4 (02) 81-89
• 11 Bourdeaux CP, Davies KJ, Thomas MJC, Bewley JS, Gould TH. Using ‘nudge’ principles for order set design: a before and after evaluation of an electronic prescribing template in critical care. BMJ Qual Saf 2014; 23 (05) 382-388
• 12 HHS Acting Secretary Declares Public Health Emergency to Address National Opioid Crisis. U.S. Department of Health and Human Services. https://www.hhs.gov/about/news/2017/10/26/hhs-acting-secretary-declares-public-health-emergency-address-national-opioid-crisis.html . Published May 23, 2018. Accessed August 15, 2018
• 13 Devlin JW, Skrobik Y, Gélinas C. , et al. Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU. Crit Care Med 2018; 46 (09) e825-e873
• 14 Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain—United States, 2016. JAMA 2016; 315 (15) 1624-1645
• 15 Chou R, Gordon DB, de Leon-Casasola OA. , et al. Management of postoperative pain: a clinical practice guideline from the American Pain Society, the American Society of Regional Anesthesia and Pain Medicine, and the American Society of Anesthesiologists' Committee on Regional Anesthesia, Executive Committee, and Administrative Council. J Pain 2016; 17 (02) 131-157
• 16 Lin PC, Chang CH, Hsu PI, Tseng PL, Huang YB. The efficacy and safety of proton pump inhibitors vs histamine-2 receptor antagonists for stress ulcer bleeding prophylaxis among critical care patients: a meta-analysis. Crit Care Med 2010; 38 (04) 1197-1205
• 17 MacLaren R, Reynolds PM, Allen RR. Histamine-2 receptor antagonists vs proton pump inhibitors on gastrointestinal tract hemorrhage and infectious complications in the intensive care unit. JAMA Intern Med 2014; 174 (04) 564-574
• 18 Azab M, Doo L, Doo DH. , et al. Comparison of the hospital-acquired Clostridium difficile infection risk of using proton pump inhibitors versus histamine-2 receptor antagonists for prophylaxis and treatment of stress ulcers: a systematic review and meta-analysis. Gut Liver 2017; 11 (06) 781-788
• 19 Bashar FR, Manuchehrian N, Mahmoudabadi M, Hajiesmaeili MR, Torabian S. Effects of ranitidine and pantoprazole on ventilator-associated pneumonia: a randomized double-blind clinical trial. Tanaffos 2013; 12 (02) 16-21
• 20 Lilly CM, Aljawadi M, Badawi O. , et al. Comparative effectiveness of proton pump inhibitors vs. histamine type-2 receptors blockers for preventing clinically important gastrointestinal bleeding during intensive care: a population-based study. Chest 2018; 154 (03) 557-566
• 21 Semler MW, Self WH, Wanderer JP. , et al; SMART Investigators and the Pragmatic Critical Care Research Group. Balanced crystalloids versus saline in critically ill adults. N Engl J Med 2018; 378 (09) 829-839
• 22 Self WH, Semler MW, Wanderer JP. , et al; SALT-ED Investigators. Balanced crystalloids versus saline in noncritically ill adults. N Engl J Med 2018; 378 (09) 819-828
• 23 Krajewski ML, Raghunathan K, Paluszkiewicz SM, Schermer CR, Shaw AD. Meta-analysis of high- versus low-chloride content in perioperative and critical care fluid resuscitation. Br J Surg 2015; 102 (01) 24-36
• 24 Serpa Neto A, Martin Loeches I, Klanderman RB. , et al; PROVE Network Investigators. Balanced versus isotonic saline resuscitation-a systematic review and meta-analysis of randomized controlled trials in operation rooms and intensive care units. Ann Transl Med 2017; 5 (16) 323
• 25 Handy JM, Soni N. Physiological effects of hyperchloraemia and acidosis. Br J Anaesth 2008; 101 (02) 141-150
• 26 Sears A, Shneiderman B. Split menus: effectively using selection frequency to organize menus. ACM Trans Comput Interact 1994; 1: 27-51
• 27 Jacobs BR, Hart KW, Rucker DW. Reduction in Clinical Variance Using Targeted Design Changes in Computerized Provider Order Entry (CPOE) order sets: impact on hospitalized children with acute asthma exacerbation. Appl Clin Inform 2012; 3 (01) 52-63
• 28 Olson J, Hollenbeak C, Donaldson K, Abendroth T, Castellani W. Default settings of computerized physician order entry system order sets drive ordering habits. J Pathol Inform 2015; 6: 16
• 29 Leis B, Frost A, Bryce R, Lyon AW, Coverett K. Altering standard admission order sets to promote clinical laboratory stewardship: a cohort quality improvement study. BMJ Qual Saf 2019; 28 (10) 846-852
• 30 McGreevey III JD. Order sets in electronic health records: principles of good practice. Chest 2013; 143 (01) 228-235
• 31 Li RC, Wang JK, Sharp C, Chen JH. When order sets do not align with clinician workflow: assessing practice patterns in the electronic health record. BMJ Qual Saf 2019; 28 (12) 987-996
• 32 Rosenbloom ST, Chiu KW, Byrne DW, Talbert DA, Neilson EG, Miller RA. Interventions to regulate ordering of serum magnesium levels: report of an unintended consequence of decision support. J Am Med Inform Assoc 2005; 12 (05) 546-553