Sustained Improvement in Inflammatory Bowel Disease Quality Measures Using an Electronic Health Record Intervention

Abstract

Background Inflammatory bowel disease (IBD) is a chronic condition with wide variation in treatment and resource utilization because of many different disease presentations and treatment options. In an effort to standardize care and improve health outcomes, several organizations have created performance measures to monitor various aspects of IBD care.

Objectives We aimed to assess longitudinal documentation adherence with physician quality reporting system's (PQRS) IBD performance measures before, immediately after, and 1 year following the implementation of a comprehensive electronic health record (EHR) IBD clinical documentation support tool intervention.

Methods We reviewed 50 patient charts that were randomly selected from consecutive outpatient IBD visits at our tertiary care center from September 1, 2015 to June 30, 2016, prior to implementation of an IBD-specific note template, order set, and patient education handout on September 1, 2016. Two additional cohorts of 50 patient charts were randomly selected from September 1, 2016 to June 30, 2017 and September 1, 2017 to June 30, 2018. These charts were reviewed to assess adherence of pertinent PQRS performance measures for outpatient IBD care. The project was deemed not human subjects research and received exempt approval by the Institutional Review Board (IRB#: IRB00040399).

Results The cohort immediately after the intervention showed significant increases in documentation rates of influenza immunization (19–59%, p < 0.001), pneumococcal immunizations (2–38%, p < 0.001), tobacco cessation (28.6–77.8%, p = 0.049), and proportion of all eligible measures (40.6–62.2%, p < 0.001) when compared with the preintervention group. Moreover, documentation rates were sustained in the 1-year follow-up group when compared with the postintervention group.

Conclusion A multifaceted, EHR focused approach can significantly and sustainably improve documentation of outpatient IBD quality measures.

Keywords

electronic health records - implementation - quality - note - order set

Background and Significance

Inflammatory bowel disease (IBD), including Crohn's disease and ulcerative colitis, is an incurable chronic condition, is often diagnosed at a young age, and can require a lifetime of medical care. Almost 1 million Americans have IBD and approximately 30% of those are diagnosed during childhood.[1] IBD can significantly impact individual health and quality of life, and it can also be of great financial burden to patients, families, and society at large. The Centers for Disease Control and Prevention estimates that IBD accrues direct medical costs of $6.3 billion each year, while indirect costs, such as missed work opportunities, contribute an additional $5.5 billion annually.[2]

Of interest, the heterogeneity of disease phenotypes and the multitude of treatment options for IBD lead to wide variation in treatment and resource utilization. Studies have shown that physicians often make different diagnoses, order various studies, and prescribe different therapies when given the same clinical data.[3] Significant variation in care is largely considered an indicator of poor quality health care.[4] The development and use of quality measures strive to standardize aspects of care for IBD and ultimately lead to improved outcomes.

Additionally, the quality of care provided can also be assessed by determining how often quality measures are being followed.[5] The Centers for Medicare and Medicaid Services (CMS) believes quality measures that are meaningful should be used to address high impact areas of disease management with significant opportunity for improvement while minimizing the level of burden for providers.[6] Furthermore, CMS has tied provider reimbursement to adherence with documenting these measures. The first IBD quality measures were introduced by the American Gastroenterological Association (AGA) in 2011. The AGA established ten quality metrics for IBD with the intent of improving health outcomes by emphasizing preventive care and noncorticosteroid-based treatment options.[4] These measures were eventually narrowed into seven performance measures that apply to outpatient IBD care for the Physician Quality Reporting System (PQRS), a quality reporting program initiated by CMS. Since that time, the landscape for quality metrics reporting has changed with the introduction of merit-based incentive payment system and advanced alternative payment models.[7] In addition, prior studies have shown adherence with IBD quality measures to be poor overall.[8] [9] [10] While it remains unclear which specific measures will be chosen for the next iteration of quality metrics reporting, sustained adherence with the measures remains essential. During this time of transition, simplified EHR improvements are useful in aiding with this shift in documentation.

Objectives

Our aim for this study was to assess documentation adherence with select PQRS performance measures at one academic medical center before and after the implementation of a multifaceted, EHR driven approach including: (1) an IBD order set; (2) note template; and (3) patient education handout. Patient encounters were evaluated both immediately following the EHR enhancements and 1 year after its implementation to assess for sustained improvement in documentation rates.

Methods

This quality improvement project was conducted at our tertiary care academic medical center, Wake Forest Baptist Medical Center (WFBMC), in Winston Salem, North Carolina. Participation was limited to two different sites within our institution: (1) an academic gastroenterology practice and (2) a fellows' clinic, where several different attending physicians and nine fellows provide care.

Outpatient encounters from September 1, 2015 to June 30, 2016 for patients aged 18 years and older with a visit diagnosis of IBD (including all applicable International Classification of Diseases [ICD]-10 codes as outlined by PQRS)[11] were included in the study. Fifty patient charts were selected for review using a random number generator, regardless of whether the encounter note utilized the IBD-specific note template, other template, or dictation software. Using Epic, the EHR system employed at WFBMC, each chart was manually reviewed by one of the authors for adherence with PQRS quality measures ([Table 1]). Of the seven PQRS measures, evaluation for adherence with the management of patients with corticosteroid-sparing therapies was excluded. Additionally, the measure involving tobacco use was evaluated for both screening and cessation separately.

Table 1 2016 PQRS measures for inflammatory bowel disease

Measure #110: Preventative care and screening: influenza immunization

Percentage of patients aged 6 months and older seen for a visit between October 1 and March 31 who received an influenza immunization or who reported previous receipt of an influenza immunization.

Measure #111: Pneumonia vaccination status

Percentage of patients 65 years of age and older who have ever received a pneumococcal vaccine.

Measure #226: Preventive care and screening: tobacco use: screening and cessation intervention

Percentage of patients aged 18 years and older who were screened for tobacco use one or more times within 24 months and who received cessation counseling intervention if identified as a tobacco user.

Measure #270: Preventive care: corticosteroid sparing therapy[a]

Percentage of patients aged 18 years and older with a diagnosis of inflammatory bowel disease (IBD) who have been managed by corticosteroids greater than or equal to 10 mg/day of prednisone equivalents for 60 or greater consecutive days or a single prescription equating to 600-mg prednisone or greater for all fills that have been prescribed corticosteroid sparing therapy within the last 12 months.

Measure #271: Preventive care: corticosteroid related iatrogenic injury—bone loss assessment

Percentage of patients aged 18 years and older with an IBD encounter who were prescribed prednisone equivalents greater than or equal to 10 mg/day for 60 or greater consecutive days or a single prescription equating to 600-mg prednisone or greater for all fills and were documented for risk of bone loss once during the reporting year or the previous calendar year.

Measure #274: Testing for latent tuberculosis (TB) before initiating anti-tumor necrosis factor (TNF) therapy

Percentage of patients aged 18 years and older with a diagnosis of IBD for whom a TB screening was performed and results interpreted within 6 months prior to receiving a first course of anti-TNF (tumor necrosis factor) therapy.

Measure #275: Assessment of hepatitis B virus (HBV) status before initiating anti-TNF (tumor necrosis factor) therapy

Percentage of patients aged 18 years and older with a diagnosis of IBD who had HBV status assessed and results interpreted within 1 year prior to receiving a first course of anti-TNF (tumor necrosis factor) therapy.

^a Measure excluded from study.

Assessing documentation adherence to each measure was performed in accordance with PQRS criteria.[11] A chart was eligible for the influenza measure if the visit occurred between October 1 and March 31. The measure was met if the encounter note documented that the patient received immunization or reported previous receipt of an influenza immunization during the current season. All patients were eligible for pneumonia vaccination measure, as only patients aged 18 years and older were included in the study. This measure was met if there was documentation in the encounter note that the pneumonia vaccine was administered or previously received. All patients in the study were eligible for tobacco screening, and cessation intervention was assessed for the tobacco users. Patients were eligible for bone loss assessment if they received 10 mg/d or greater of prednisone equivalents for 60 or more consecutive days or if prescribed 600 mg or greater of prednisone equivalents. Eligible charts were reviewed for central dual-energy X-ray absorptiometry (DXA) in the preceding 2 years. Regarding patients with recently initiated anti-tumor necrosis factor (TNF) therapy, a patient was considered for latent tuberculosis (TB) and hepatitis B measures if the therapy was started within 1 year of the encounter date. This was done because of the low number of visits for initiation of anti-TNF therapy. Latent TB and hepatitis B screening measures were fulfilled if testing was performed within 6 months and 1 year of anti-TNF initiation, respectively.

After the baseline data was collected, an IBD note template ([Fig. 1]), order set ([Fig. 2]), and patient education handout were created. The IBD-specific note template provided prepopulated sections and nonmodal clinical decision support (CDS) for information pertaining to the measures. Nonmodal CDS does not prevent users from interacting with the rest of the EHR. This modality was selected so to not interfere with the workflows of different providers, but to also serve as a prompt to include information relevant to the quality measures. The dynamic IBD order-set, organized with single-click diagnosis association and placement of orders, offered a simplified approach for commonly prescribed medicines, tests, procedures, and referrals for IBD care. The order set also provided reminders of important PQRS measures with one-click access to vaccines, DXA, TB, and hepatitis B testing. Furthermore, with the use of the order set, a patient education handout containing basic health maintenance information regarding vaccinations, bone health, cancer screening, and tobacco cessation resources was autopopulated into the after-visit summary (AVS) provided to the patient ([Fig. 3]). Review of the baseline adherence data and a tutorial for the IBD note template, order set, and patient education handout were discussed with faculty and fellows during a section conference prior to the second phase of the study. Additionally, emails outlining the EHR changes were sent to those not in attendance.

Fig. 1 Portion of IBD note template with nonmodal CDS for many PQRS quality @IMM@ inserts vaccination history for the patient. Nonmodal CDS denoted as { } does not interfere with workflow, but prompts provider include information prior to signing note. CDS, clinical decision support; IBD, inflammatory bowel disease; PQRS, physician quality reporting system.

Fig. 2 Screenshot of inflammatory bowel disease order set.

Fig. 3 Patient education handout.

Following these interventions, another 50 patient charts were randomly selected from outpatient IBD visits occurring between September 1, 2016 and June 30, 2017, just after the changes were implemented on September 1, 2016. Using the same methods described above, an additional 50 charts were reviewed for encounters between September 1, 2017 and June 30, 2018 to assess for a sustained response.

Categorical variables are described as frequencies and proportions, while continuous variables are described as medians and ranges. SAS 9.4 statistical software was used to analyze data described below. Analysis of categorical variables was performed using chi-square analysis or Fisher's exact test when sample sizes were less than five. p-Values less than 0.05 were considered statistically significant. Demographic data containing continuous variables were analyzed using Kruskal–Wallis test, whereas categorical variables were analyzed using a 3 × 2 chi-square analysis.

All authors had access to the data and approved the final manuscript. The project was deemed not human subjects research and received exempt approval by the Institutional Review Board at WFBMC (IRB#: IRB00040399). Additionally, the EHR screenshots included were approved for distribution by Epic Systems Corporation.

Results

Demographics

This study analyzed 150 different patient encounters, with each study group consisting of 50 different patients. There were no significant differences in age among the groups. The proportion of males in each group varied from 38 to 56%, but was not significantly different, (p = 0.195). While the preintervention group had the highest proportion of patient with ulcerative colitis (48%), there was no statistically significant difference in subtype of IBD among the groups. See [Table 2] for additional demographic information.

Table 2 Demographic characteristics of each study group

	Preintervention group (n = 50)	Postintervention group (n = 50)	1-year follow-up group (n = 50)	p-Value
Median age, y (range)	46 (18–87)	46 (22–84)	41 (19–82)	0.41
Males, n (%)	23 (46)	28 (56)	19 (38)	0.195
Median BMI, kg/m2 (range)	27.1 (16–49)	26.1 (17–55)	26.4 (12–56)	0.893
IBD type, n (%)				0.122
Crohn's disease	26 (52)	35 (70)	34 (68)
Ulcerative colitis	24 (48)	15 (30)	16 (32)
Provider type, n (%)				0.846
Attending physician	44 (88)	43 (86)	42 (84)
Gastroenterology fellow	6 (12)	7 (14)	8 (16)

Abbreviations: BMI, body mass index; IBD, inflammatory bowel disease.

Comparing Adherence Between Pre- and Postintervention Groups

Preintervention and postintervention rates of adherence with PQRS measures studied are presented in [Table 3]. Baseline adherence for all evaluated quality measures was 40.6%, with high variation in adherence rates for each specific measure in the preintervention group. When compared with the preintervention group, the cohort immediately after the EHR changes were implemented revealed significant increases in EHR documentation rates of influenza immunization (19–59%, p < 0.001), pneumococcal immunizations (2–38%, p < 0.001), tobacco cessation (28.6–77.8%, p = 0.049), and proportion of all eligible measures (40.6–62.2%, p < 0.001). While sample sizes were low for latent TB and hepatitis B measures (n = 3, n = 1, respectively), there was no significant difference in rates of documentation adherence.

Table 3 Adherence with PQRS measures before and after EHR changes

Measure	Preintervention group (n = 50)	Postintervention group (n = 50)	Delta (%)	p-Value
Influenza immunization, n (%)	7/36 (19.4)	22/37 (59.5)	40	<0.001
Pneumococcal immunization, n (%)	1/50 (2)	19/50 (38)	36	<0.001
Tobacco screening, n (%)	50/50 (100)	50/50 (100)	0	1
Tobacco cessation, n (%)	2/7 (28.6)	7/9 (77.8)	49	0.049
Bone loss assessment, n (%)	0/11 (0)	2/16 (12.5)	12	0.223
Latent TB testing before anti-TNF-α therapy, n (%)	2/3 (66.7)	1/1 (100)	33	1
HBV testing before anti-TNF-α therapy, n (%)	3/3 (100)	1/1 (100)	0	1
Total combined measures, n (%)	65/160 (40.6)	102/164 (62.2)	22	<0.001

Abbreviations: EHR, electronic health record; HBV, hepatitis B virus; PQRS, physician quality reporting system; TB, tuberculosis; TNF, tumor necrosis factor.

Assessing for Sustained Adherence

[Table 4] presents the proportion of PQRS measures appropriately documented in the 1-year follow-up group when compared with the postintervention group. Charts were evaluated for sustained adherence using randomly selected encounters that occurred 1 year after the postintervention group. No additional interventions or alterations to the EHR occurred between the two study periods. While documentation adherence rates did not decrease in the 1-year follow-up group, there were significant increases in bone loss assessment (12.5–62.5%, p = 0.011) and influenza immunization (59.5–85%, p = 0.047) when compared with the postintervention group.

Table 4 Adherence with PQRS measures after EHR changes and 1-year follow-up

Measure	Postintervention group (n = 50)	1-year follow-up group (n = 50)	Delta (%)	p-Value
Influenza immunization, n (%)	22/37 (59.5)	17/20 (85)	26	0.047
Pneumococcal immunization, n (%)	19/50 (38)	19/50 (38)	0	1
Tobacco screening, n (%)	50/50 (100)	50/50 (100)	0	1
Tobacco cessation, n (%)	7/9 (77.8)	3/4 (75)	−3	1
Bone loss assessment, n (%)	2/16 (12.5)	5/8 (62.5)	50	0.011
Latent TB testing before anti-TNF-α therapy, n (%)	1/1 (100)	5/6 (83.3)	−17	1
HBV testing before anti-TNF-α therapy, n (%)	1/1 (100)	5/6 (83.3)	−17	1
Total combined measures, n (%)	102/164 (62.2)	104/144 (72.2)	10	0.062

Abbreviations: EHR, electronic health record; HBV, hepatitis B virus; PQRS, physician quality reporting system; TB, tuberculosis; TNF, tumor necrosis factor.

Discussion

This study suggests that simple improvements to the EHR via streamlined order set, note template, and patient education handout may contribute to sustained improvements in documentation adherence of outpatient IBD quality measures. Prior to the implementation of the EHR changes, the aggregate baseline adherence rate for documentation of 2016 PQRS measures at our academic institution was 40.6% of total eligible measures. Adherence with individual measures ranged widely from 0 to 100%, for bone loss assessment and screening for hepatitis B prior to initiation of anti-TNF α therapy, respectively. Many prior studies have shown infrequent and inconsistent adherence with PQRS measures for IBD.[8] [9] [10] One study reported rates as little as 3% for pneumococcal immunization to 98% for corticosteroid-sparing therapy.[12] As previously described, wide variation in care is considered to be an indicator of low quality care.[4] Hence, wide variation in adherence with quality measures may also represent poor care.

While there was low baseline documentation adherence, there was robust adherence with tobacco screening. This was thought to be due to prior process improvement measures at our institution, where nursing staff is prompted to document the information in a specific location in the EHR during the patient intake process. It was postulated that a similar physician-oriented process for other measures would prove effective in improving overall adherence in documentation. Additionally, it was felt that easier access to the studies utilized in each quality measure would improve efficiency and consequently improve adherence. Thus, an IBD note template, order set, and patient education handout were designed to improve documentation adherence with PQRS measures.

The initial comparison between pre- and postintervention groups showed significant increases in documentation adherence rates of influenza and pneumococcal vaccine measures. One benefit of the implemented order set includes a section for vaccinations, where orders for influenza, pneumococcal, and other immunizations can easily be placed. Utilizing the order set serves as an inherent, secondary reminder for the clinician to document quality measures. Additionally, use of the order set automatically populates the patient's AVS with education materials, including resources for smoking cessation. Therefore, use of the order set passively leads to adherence with the smoking cessation measure, as seen with the significant increase in tobacco cessation adherence. While latent TB and hepatitis B screening measures had robust adherence at baseline, there was no statistically significant change after the improvements to the EHR. This was likely attributable to the small number of eligible patients for each measure (n = 3, n = 1, respectively), as few patients initiated anti-TNF α therapy within the preceding year.

While assessing for sustained adherence, there were no significant decreases in adherence rates when comparing the 1-year follow-up group to the postintervention group. There were, however, significant increases in bone loss assessment and influenza immunization measures. Increased adherence with the influenza measure reflects our health system's efforts to stress the importance of vaccination to providers during influenza season. The rise in adherence to the bone loss assessment measure can be attributed to the criteria of the measure itself. CMS states that the measure is met if a DXA scan was ordered or performed in the preceding 2 years.[11] Thus, the delayed increase in adherence during the 1-year follow-up group (occurring 12–21 months after the EHR changes) is likely due to the inclusion of patients with more postintervention encounters occurring in the preceding 2 years.

Our study is unique due to its demonstrated sustained overall improvement in adherence to documentation of quality measures. Prior studies have shown that bundle-based EHR interventions led to sustained reductions in pediatric ventilator-associated pneumonia and surgical site infections.[13] [14] However, to our knowledge, published data on IBD performance measures have not evaluated adherence beyond the initial implementation period. One aspect of our EHR interventions that may have improved the sustainability is the modifiability of the note template. Faculty were encouraged to use either the entire IBD note template or add the PQRS measures portion ([Fig. 1]) into existing personalized note templates, making it easier to implement into individualized workflows. Although this study was not designed to assess for usage of the note template, it was noted that performance measures were typically missed when the template was not utilized. Additional barriers to address in the future include how to increase the fidelity of provider use in using the EHR interventions.

While assessing for improvements in documentation of IBD quality measures through the use of a note template has been studied in the past,[15] coupling the template with an order set has not been investigated. Prior studies suggest the implementation of CDS and order sets in different clinical environments has improved efficiency and outcomes and decreased ordering errors and treatment variance.[16] [17] [18] [19] The order set implemented in this study provides physicians in our institution with an efficient way to order studies, medicines, procedures, and referrals common to IBD encounters ([Fig. 2]). The simplicity of ordering the specific tests associated with PQRS quality measures by using a limited number of clicks likely led to improved stakeholder buy-in and sustained culture change at our institution. This approach highlights the useful framework of the CDS Five Rights model, as outlined by the Agency for Healthcare Research and Quality, by utilizing the right intervention format through the right channel at the right time in the workflow.[20] Additionally, the order set also serves as an additional reminder for providers to explore whether quality measures have been completed.

A chasm exists between improved documentation of quality measures and a realized improvement in clinical practice. One limitation of our study is that the design strictly focused on documentation of quality measures, but did not investigate differences in the frequency of tests or therapies being ordered or completed. In theory, increased documentation or increased prompting for documentation of quality measures may lead to increased completion of the quality measures, but this needs to be studied. There may be patient encounters where quality measures were completed, but not documented, leading to no additional benefit to the patient. Quality measures were designed to standardize and ultimately improve care and outcomes for patients with IBD. Future studies are needed to evaluate the differences in actual completion of appropriate tests with utilization of the IBD order set, as well as changes to patient outcomes.

Additionally, the design of the study limits the ability to detect underlying secular trends that may have influenced changes in the documentation of quality measures. While our study team was not aware of coexisting initiatives comprehensively addressing vaccinations, IBD, documentation, or quality measures at our institution during the study period, it remains unknown if external factors influenced the results. Future studies will strive to incorporate a control group into the design.

With recent changes to reimbursement models, compensation is now tied to documentation adherence through the use of quality measures. The simplified approach and EHR changes implemented in this study were designed to help satisfy quality measures while also improving efficiency. While this study demonstrates sustained improvements within the gastroenterology section of a tertiary academic center, similar interventions may lead to improved documentation of quality measures when applied to other practice settings or chronic conditions such as rheumatoid arthritis or chronic kidney disease. Because there is constant change in specific quality measures, yearly updates to the note template may be helpful in the long-term for continued adherence with up-to-date measures.

Conclusion

A multifaceted, EHR focused approach utilizing an IBD note template, order set, and patient education handout contributed to significant and sustained improvements in documentation and completion of outpatient IBD quality measures.

Clinical Relevance Statement

We reported on the impact of an electronic health record bundle specifically designed for outpatient encounters with patients who suffer from inflammatory bowel disease. Our findings suggest that combining a note template, order set, and patient education handout leads to improvements in the rates of documentation of specific inflammatory bowel disease quality measures that are sustained.

Multiple Choice Questions

1. Wide variation in the treatment of inflammatory bowel disease can be attributable to:

   • The heterogeneity of disease phenotypes.

   • The multitude of treatment options.

   • Inconsistent resource utilization.

   • All of the above.

   Correct Answer: The correct answer is option d, all of the above.

2. Quality measures are intended to:

   • Standardize aspects of care with significant opportunity for improvement.

   • Increase the level burden for providers.

   • Increase the costs of health care.

   • Have no effect on health outcomes.

   Correct Answer: The correct answer is option a, standardize aspects of care with significant opportunity for improvement.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

The project was deemed not human subjects research and received exempt approval by the Institutional Review Board at WFBMC (IRB no: IRB00040399).

• References

• 1 Kappelman MD, Rifas-Shiman SL, Kleinman K. , et al. The prevalence and geographic distribution of Crohn's disease and ulcerative colitis in the United States. Clin Gastroenterol Hepatol 2007; 5 (12) 1424-1429
• 2 Centers for Disease Control and Prevention. Inflammatory bowel disease is an expensive disease. Available at: https://www.cdc.gov/ibd/pdf/inflammatory-bowel-disease-an-expensive-disease.pdf . Accessed February 28, 2019
• 3 Esrailian E, Spiegel BMR, Targownik LE, Dubinsky MC, Targan SR, Gralnek IM. Differences in the management of Crohn's disease among experts and community providers, based on a national survey of sample case vignettes. Aliment Pharmacol Ther 2007; 26 (07) 1005-1018
• 4 Weaver KN, Kappelman MD, Sandler RS. , et al. Variation in care of inflammatory bowel diseases patients in Crohn's and Colitis Foundation of America Partners: role of gastroenterologist practice setting in disease outcomes and quality process measures. Inflamm Bowel Dis 2016; 22 (11) 2672-2677
• 5 Berry SK, Siegel CA, Melmed GY. Quality improvement initiatives in inflammatory bowel disease. Curr Gastroenterol Rep 2017; 19 (08) 41
• 6 Moody-Williams J, Yong P, Long T. Meaningful measures—November 2017. Available at: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/Downloads/Meaningful-Measures-webinar-slides-11-30-17.pdf . Accessed February 28, 2019
• 7 McConnell R, Velayos F. The vanishing tide: as MACRA moves in, IBD quality measures move out. Available at: https://www.mdedge.com/gihepnews/article/156121/vanishing-tide-macra-moves-ibd-quality-measures-move-out . Accessed March 18, 2019
• 8 Feuerstein JD, Lewandowski JJ, Martinez-Vazquez M, Leffler DA, Cheifetz AS. Documented compliance with inflammatory bowel disease quality measures is poor. Dig Dis Sci 2015; 60 (02) 339-344
• 9 Feuerstein JD, Castillo NE, Siddique SS. , et al. Poor documentation of inflammatory bowel disease quality measures in academic, community, and private practice. Clin Gastroenterol Hepatol 2016; 14 (03) 421-428
• 10 Greene L, Moreo K. Quality improvement education to improve performance on ulcerative colitis quality measures and care processes aligned with National Quality Strategy priorities. BMJ Qual Improv Rep 2015; 4 (01) u208829.w3554
• 11 Centers for Medicare and Medicaid Services. 2016 PQRS measures group specification manual. Available at: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/PQRS/downloads/2016_PQRS_IndivMeasures_Guide_11_17_2015.pdf . Accessed February 28, 2019
• 12 Sapir T, Moreo K, Carter JD, Greene L, Patel B, Higgins PD. Continuing medical education improves gastroenterologists' compliance with inflammatory bowel disease quality measures. Dig Dis Sci 2016; 61 (07) 1862-1869
• 13 McBeth CL, Montes RS, Powne A, North SE, Natale JE. Interprofessional approach to the sustained reduction in ventilator-associated pneumonia in a pediatric intensive care unit. Crit Care Nurse 2018; 38 (06) 36-45
• 14 Russell TA, Chung H, Riad C. , et al. Sustaining improvement: implementation and spread of a surgical site infection bundle. Am Surg 2018; 84 (10) 1665-1669
• 15 Feuerstein JD, Papamichael K, Popejoy S. , et al. Targeted physician education and standardizing documentation improves documented reporting with inflammatory bowel disease quality measures in a large academic and private practice. Dig Dis Sci 2018; 63 (01) 36-45
• 16 Yarahuan JW, Billet A, Hron JD. A quality improvement initiative to decrease platelet ordering errors and a proposed model for evaluating clinical decision support effectiveness. Appl Clin Inform 2019; 10 (03) 505-512
• 17 Fargo EL, D'Amico F, Pickering A, Fowler K, Campbell R, Baumgartner M. Impact of electronic physician order-set on antibiotic ordering time in septic patients in the emergency department. Appl Clin Inform 2018; 9 (04) 869-874
• 18 Jacobs BR, Hart KW, Rucker DW. Reduction in clinical variance using targeted design changes in computerized provider order entry (CPOE) order sets: impact on hospitalized children with acute asthma exacerbation. Appl Clin Inform 2012; 3 (01) 52-63
• 19 Nichols KR, Petschke AL, Webber EC, Knoderer CA. Comparison of antibiotic dosing before and after implementation of an electronic order set. Appl Clin Inform 2019; 10 (02) 229-236
• 20 Agency for Healthcare Research and Quality. Overview of CDS five rights. Available at: https://healthit.ahrq.gov/ahrq-funded-projects/current-health-it-priorities/clinical-decision-support-cds/chapter-1-approaching-clinical-decision/section-2-overview-cds-five-rights . Accessed March 18, 2019