An Electronic Medical Record-Derived Individualized Performance Metric to Measure Risk-Adjusted Adherence with Perioperative Prophylactic Bundles for Health Care Disparity Research and Implementation Science

Abstract

Background Health care disparity persists despite vigorous countermeasures. Clinician performance is paramount for equitable care processes and outcomes. However, precise and valid individual performance measures remain elusive.

Objectives We sought to develop a generalizable, rigorous, risk-adjusted metric for individual clinician performance (MIP) derived directly from the electronic medical record (EMR) to provide visual, personalized feedback.

Methods We conceptualized MIP as risk responsiveness, i.e., administering an increasing number of interventions contingent on patient risk. We embedded MIP in a hierarchical statistical model, reflecting contemporary nested health care delivery. We tested MIP by investigating the adherence with prophylactic bundles to reduce the risk of postoperative nausea and vomiting (PONV), retrieving PONV risk factors and prophylactic antiemetic interventions from the EMR. We explored the impact of social determinants of health on MIP.

Results We extracted data from the EMR on 25,980 elective anesthesia cases performed at Penn State Milton S. Hershey Medical Center between June 3, 2018 and March 31, 2019. Limiting the data by anesthesia Current Procedural Terminology code and to complete cases with PONV risk and antiemetic interventions, we evaluated the performance of 83 anesthesia clinicians on 2,211 anesthesia cases. Our metric demonstrated considerable variance between clinicians in the adherence to risk-adjusted utilization of antiemetic interventions. Risk seemed to drive utilization only in few clinicians. We demonstrated the impact of social determinants of health on MIP, illustrating its utility for health science and disparity research.

Conclusion The strength of our novel measure of individual clinician performance is its generalizability, as well as its intuitive graphical representation of risk-adjusted individual performance. However, accuracy, precision and validity, stability over time, sensitivity to system perturbations, and acceptance among clinicians remain to be evaluated.

Keywords

health care disparity - perioperative care - postoperative nausea and vomiting - implementation science - electronic medical record

Protection of Human and Animal Subjects

The Penn State Health Institutional Review Board determined that the described work met the criteria for exempt research according to the policies of this institution and the provisions of applicable federal regulations (STUDY00007035).

• References

• 1 Link BG, Phelan J. Understanding racial and ethnic disparities in postsurgical complications occurring in U.S. hospitals. J Health Soc Behav 1995; (Spec No): 80-94
• 2 Witt WP, Coffey RM, Lopez-Gonzalez L. , et al. Understanding racial and ethnic disparities in postsurgical complications occurring in U.S. hospitals. Health Serv Res 2017; 52 (01) 220-243
• 3 Krieger N, Williams DR, Moss NE. Measuring social class in US public health research: concepts, methodologies, and guidelines. Annu Rev Public Health 1997; 18: 341-378
• 4 Andreae MH, Andreae DA, Maman S. The ethics of universal health insurance. Anesth Analg 2020; 130 (03) e85-e87
• 5 Andreae MH, Gabry JS, Goodrich B, White RS, Hall C. Antiemetic prophylaxis as a marker of health care disparities in the national anesthesia clinical outcomes registry. Anesth Analg 2018; 126 (02) 588-599
• 6 Thorlby R, Jorgensen S, Siegel B, Ayanian JZ. How health care organizations are using data on patients' race and ethnicity to improve quality of care. Milbank Q 2011; 89 (02) 226-255
• 7 Andreae MH, White RS, Chen KY, Nair S, Hall C, Shaparin N. The effect of initiatives to overcome language barriers and improve attendance: a cross-sectional analysis of adherence in an inner city chronic pain clinic. Pain Med 2017; 18 (02) 265-274
• 8 Sequist TD, Fitzmaurice GM, Marshall R, Shaykevich S, Safran DG, Ayanian JZ. Physician performance and racial disparities in diabetes mellitus care. Arch Intern Med 2008; 168 (11) 1145-1151
• 9 Silber JH, Kennedy SK, Even-Shoshan O. , et al. Anesthesiologist direction and patient outcomes. Anesthesiology 2000; 93 (01) 152-163
• 10 Gaba DM. Anaesthesiology as a model for patient safety in health care. BMJ 2000; 320 (7237): 785-788
• 11 Atchabahian A, Andreae M. Long-term functional outcomes after regional anesthesia: a summary of the published evidence and a recent cochrane review. Refresh Courses Anesthesiol 2015; 43 (01) 15-26
• 12 Maman S, Andreae M. 3025 individual anesthesia provider performance assessment. J Clin Transl Sci 2019; 3 (S1): 147 . Doi: 10.1017/cts.2019.334
• 13 Skochelak SE, Hammoud MM, Lomis KD, Borkan JM, Gonzalo JD, Lawson LE, Starr SR. , eds. Health Systems Science, 2nd ed. Philadelphia, PA: Elsevier; 2020
• 14 Kawachi I, Daniels N, Robinson DE. Health disparities by race and class: why both matter. Health Aff (Millwood) 2005; 24 (02) 343-352
• 15 Epstein RH, Dexter F, Patel N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesth Analg 2015; 121 (03) 678-692
• 16 Chau A, Ehrenfeld JM. Using real-time clinical decision support to improve performance on perioperative quality and process measures. Anesthesiol Clin 2011; 29 (01) 57-69
• 17 Hyder JA, Hebl JR. Performance measurement to demonstrate value. Anesthesiol Clin 2015; 33 (04) 679-696
• 18 Hyder JA, Niconchuk J, Glance LG. , et al. What can the national quality forum tell us about performance measurement in anesthesiology?. Anesth Analg 2015; 120 (02) 440-448
• 19 Palmer RC, Ismond D, Rodriquez EJ, Kaufman JS. Social determinants of health: Future directions for health disparities research. Am J Public Health 2019; 109 (S1): S70-S71
• 20 Bazemore AW, Cottrell EK, Gold R. , et al. “Community vital signs”: incorporating geocoded social determinants into electronic records to promote patient and population health. J Am Med Inform Assoc 2016; 23 (02) 407-412
• 21 Kheterpal S. Clinical research using an information system: the multicenter perioperative outcomes group. Anesthesiol Clin 2011; 29 (03) 377-388
• 22 Gan TJ, Diemunsch P, Habib AS. , et al; Society for Ambulatory Anesthesia. Consensus guidelines for the management of postoperative nausea and vomiting. Anesth Analg 2014; 118 (01) 85-113
• 23 Junger A, Hartmann B, Benson M. , et al. The use of an anesthesia information management system for prediction of antiemetic rescue treatment at the postanesthesia care unit. Anesth Analg 2001; 92 (05) 1203-1209
• 24 Shwartz M, Restuccia JD, Rosen AK. Composite measures of health care provider performance: a description of approaches. Milbank Q 2015; 93 (04) 788-825
• 25 Maman S, Andreae M. An EMR-derived individual anesthesia provider metric to measure risk-adjusted adherence with perioperative prophylactic bundles for health systems science and disparity research. Paper presented at: Annual Meeting of the Association of University Anesthesiologists; May 16–20, 2019; Montreal, Canada
• 26 Kappen TH, Vergouwe Y, van Wolfswinkel L, Kalkman CJ, Moons KGM, van Klei WA. Impact of adding therapeutic recommendations to risk assessments from a prediction model for postoperative nausea and vomiting. Br J Anaesth 2015; 114 (02) 252-260
• 27 Biedler A, Wermelt J, Kunitz O. , et al. A risk adapted approach reduces the overall institutional incidence of postoperative nausea and vomiting. Can J Anaesth 2004; 51 (01) 13-19
• 28 Kumar A, Brampton W, Watson S, Reid VL, Neilly D. Postoperative nausea and vomiting: simple risk scoring does work. Eur J Anaesthesiol 2012; 29 (01) 57-59
• 29 Rüsch D, Eberhart L, Biedler A, Dethling J, Apfel CC. Prospective application of a simplified risk score to prevent postoperative nausea and vomiting. Can J Anaesth 2005; 52 (05) 478-484
• 30 Frenzel JC, Kee SS, Ensor JE, Riedel BJ, Ruiz JR. Ongoing provision of individual clinician performance data improves practice behavior. Anesth Analg 2010; 111 (02) 515-519
• 31 Kooij FO, Klok T, Hollmann MW, Kal JE. Automated reminders increase adherence to guidelines for administration of prophylaxis for postoperative nausea and vomiting. Eur J Anaesthesiol 2010; 27 (02) 187-191
• 32 Bowen DJ, Kreuter M, Spring B. , et al. How we design feasibility studies. Am J Prev Med 2009; 36 (05) 452-457
• 33 von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. ; STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 2007; 335 (7624): 806-808
• 34 Macario A, Chung A, Weinger MB. Variation in practice patterns of anesthesiologists in California for prophylaxis of postoperative nausea and vomiting. J Clin Anesth 2001; 13 (05) 353-360
• 35 Duran D, Asada Y, Millum J, Gezmu M. Harmonizing health disparities measurement. Am J Public Health 2019; 109 (S1): S25-S27
• 36 Wickham H. Ggplot2: Elegant Graphics for Data Analysis. Vienna, Austria: R Foundation for Statistical Computing; ; New York, NY: Springer; 2016
• 37 Wickham H, François R, Henry L, Müller K. Dplyr: A Grammar of Data Manipulation. Vienna, Austria: R Foundation for Statistical Computing; 2019
• 38 Wickham H, Bryan J. Readxl: Read Excel Files. Vienna, Austria: R Foundation for Statistical Computing; ; New York, NY: Springer; 2019
• 39 Xie Y. Knitr: A General-Purpose Package for Dynamic Report Generation in R. Vienna, Austria: R Foundation for Statistical Computing; ; New York, NY: Springer; 2019
• 40 Xu HF, White RS, Sastow DL, Andreae MH, Gaber-Baylis LK, Turnbull ZA. Medicaid insurance as primary payer predicts increased mortality after total hip replacement in the state inpatient databases of California, Florida and New York. J Clin Anesth 2017; 43: 24-32
• 41 Cooper LA, Hill MN, Powe NR. Designing and evaluating interventions to eliminate racial and ethnic disparities in health care. J Gen Intern Med 2002; 17 (06) 477-486
• 42 Kaye AD, Okanlawon OJ, Urman RD. Clinical performance feedback and quality improvement opportunities for perioperative physicians. Adv Med Educ Pract 2014; 5: 115-123
• 43 Maman S, Andreae M. Quantifying the impact of physician supervision on nurse anesthetists' risk-adjusted antiemetic prophylaxis. Paper presented at: Annual Meeting of the International Anesthesia Research Society; May 16–20, 2019; Montreal, Canada
• 44 Kolarczyk LM, Arora H, Manning MW, Zvara DA, Isaak RS. Defining value-based care in cardiac and vascular anesthesiology: the past, present, and future of perioperative cardiovascular care. J Cardiothorac Vasc Anesth 2018; 32 (01) 512-521
• 45 Apfel CC, Bacher A, Biedler A. , et al. A factorial trial of six interventions for the prevention of postoperative nausea and vomiting [in German]. Anaesthesist 2005; 54 (03) 201-209
• 46 Macario A, Weinger M, Carney S, Kim A. Which clinical anesthesia outcomes are important to avoid? The perspective of patients. Anesth Analg 1999; 89 (03) 652-658
• 47 Chung F, Mezei G. Factors contributing to a prolonged stay after ambulatory surgery. Anesth Analg 1999; 89 (06) 1352-1359
• 48 Eggleton K, Liaw W, Bazemore A. Impact of gaps in merit-based incentive payment system measures on marginalized populations. Ann Fam Med 2017; 15 (03) 255-257
• 49 Anderson AC, O'Rourke E, Chin MH, Ponce NA, Bernheim SM, Burstin H. Promoting health equity and eliminating disparities through performance measurement and payment. Health Aff (Millwood) 2018; 37 (03) 371-377
• 50 Glance LG, Neuman M, Martinez EA, Pauker KY, Dutton RP. Performance measurement at a “tipping point”. Anesth Analg 2011; 112 (04) 958-966
• 51 Peccora CD, Gimlich R, Cornell RP, Vacanti CA, Ehrenfeld JM, Urman RD. Anesthesia report card - a customizable tool for performance improvement. J Med Syst 2014; 38 (09) 105
• 52 Multicenter Perioperative Outcomes Group. (2020, July 14). Postoperative Nausea and Vomiting PONV-01. ASPIRE Measures. Available at: https://spec.mpog.org/Spec/Public/24
• 53 Jeyabalan S, Thampi SM, Karuppusami R, Samuel K. Comparing the efficacy of aprepitant and ondansetron for the prevention of postoperative nausea and vomiting (PONV): a double blinded, randomised control trial in patients undergoing breast and thyroid surgeries. Indian J Anaesth 2019; 63 (04) 289-294
• 54 Bartlett R, Hartle AJ. Routine use of dexamethasone for postoperative nausea and vomiting: the case against. Anaesthesia 2013; 68 (09) 892-896
• 55 McDonnell C, Barlow R, Campisi P, Grant R, Malkin D. Fatal peri-operative acute tumour lysis syndrome precipitated by dexamethasone. Anaesthesia 2008; 63 (06) 652-655
• 56 Frasier LL, Pavuluri Quamme SR, Ma Y. , et al. Familiarity and communication in the operating room. J Surg Res 2019; 235: 395-403
• 57 Sintchenko V, Magrabi F, Tipper S. Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review. Med Inform Internet Med 2007; 32 (03) 225-240
• 58 Kooij FO, Klok T, Hollmann MW, Kal JE. Decision support increases guideline adherence for prescribing postoperative nausea and vomiting prophylaxis. Anesth Analg 2008; 106 (03) 893-898
• 59 Simpao AF, Tan JM, Lingappan AM, Gálvez JA, Morgan SE, Krall MA. A systematic review of near real-time and point-of-care clinical decision support in anesthesia information management systems. J Clin Monit Comput 2017; 31 (05) 885-894
• 60 Chien AT, Chin MH, Davis AM, Casalino LP. Pay for performance, public reporting, and racial disparities in health care: how are programs being designed?. Med Care Res Rev 2007; 64 (5, Suppl): 283S-304S
• 61 Warner DO, Isaak RS, Peterson-Layne C. , et al. Development of an objective structured clinical examination as a component of assessment for initial board certification in anesthesiology. Anesth Analg 2020; 130 (01) 258-264
• 62 Colquhoun DA, Shanks AM, Kapeles SR. , et al. Considerations for integration of perioperative electronic health records across institutions for research and quality improvement: the approach taken by the multicenter perioperative outcomes group. Anesth Analg 2020; 130 (05) 1133-1146
• 63 McCormick PJ, Yeoh C, Vicario-Feliciano RM. , et al. Improved compliance with anesthesia quality measures after implementation of automated monthly feedback. J Oncol Pract 2019; 15 (06) e583-e592
• 64 Tetzlaff JE. Assessment of competency in anesthesiology. Anesthesiology 2007; 106 (04) 812-825
• 65 Kooij FO, Klok T, Preckel B, Hollmann MW, Kal JE. The effect of requesting a reason for non-adherence to a guideline in a long running automated reminder system for PONV prophylaxis. Appl Clin Inform 2017; 8 (01) 313-321
• 66 Hoberman J. Medical racism and the rhetoric of exculpation: how do physicians think about race?. New Lit Hist 2007; 38 (03) 505-525
• 67 Andreae MH, Nair S, Gabry JS, Goodrich B, Hall C, Shaparin N. A pragmatic trial to improve adherence with scheduled appointments in an inner-city pain clinic by human phone calls in the patient's preferred language. J Clin Anesth 2017; 42: 77-83
• 68 Shaparin N, White R, Andreae M, Hall C, Kaufman A. A longitudinal linear model of patient characteristics to predict failure to attend an inner-city chronic pain clinic. J Pain 2014; 15 (07) 704-711
• 69 Payne VL, Hysong SJ. Model depicting aspects of audit and feedback that impact physicians' acceptance of clinical performance feedback. BMC Health Serv Res 2016; 16: 260
• 70 Carpenter B, Gelman A, Hoffman M. , et al. Stan: a probabilistic programming language. J Stat Softw 2017; 76 (01) 1-32
• 71 Bürkner P-C. Advanced Bayesian multilevel modeling with the R package brms. R J 2018; 10 (01) 395-411