Download PDF
Appl Clin Inform 2020; 11(04): 570-577
DOI: 10.1055/s-0040-1715827
Research Article

Implementation of Artificial Intelligence-Based Clinical Decision Support to Reduce Hospital Readmissions at a Regional Hospital

Santiago Romero-Brufau
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
2   Department of Biostatistics, Harvard T.H. Chan School of Public Health, Harvard University, Cambridge, Massachusetts, United States
,
Kirk D. Wyatt
3   Division of Pediatric Hematology/Oncology, Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, Minnesota, United States
,
Patricia Boyum
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
,
Mindy Mickelson
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
,
Matthew Moore
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
,
Cheristi Cognetta-Rieke
4   Department of Nursing, Mayo Clinic Health System, La Crosse, La Crosse, Wisconsin, United States
› Author Affiliations Funding This study received its financial support from Mayo Clinic research funds.
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background Hospital readmissions are a key quality metric, which has been tied to reimbursement. One strategy to reduce readmissions is to direct resources to patients at the highest risk of readmission. This strategy necessitates a robust predictive model coupled with effective, patient-centered interventions.

Objective The aim of this study was to reduce unplanned hospital readmissions through the use of artificial intelligence-based clinical decision support.

Methods A commercially vended artificial intelligence tool was implemented at a regional hospital in La Crosse, Wisconsin between November 2018 and April 2019. The tool assessed all patients admitted to general care units for risk of readmission and generated recommendations for interventions intended to decrease readmission risk. Similar hospitals were used as controls. Change in readmission rate was assessed by comparing the 6-month intervention period to the same months of the previous calendar year in exposure and control hospitals.

Results Among 2,460 hospitalizations assessed using the tool, 611 were designated by the tool as high risk. Sensitivity and specificity for risk assignment were 65% and 89%, respectively. Over 6 months following implementation, readmission rates decreased from 11.4% during the comparison period to 8.1% (p < 0.001). After accounting for the 0.5% decrease in readmission rates (from 9.3 to 8.8%) at control hospitals, the relative reduction in readmission rate was 25% (p < 0.001). Among patients designated as high risk, the number needed to treat to avoid one readmission was 11.

Conclusion We observed a decrease in hospital readmission after implementing artificial intelligence-based clinical decision support. Our experience suggests that use of artificial intelligence to identify patients at the highest risk for readmission can reduce quality gaps when coupled with patient-centered interventions.

Keywords

intelligence - artificial - patient readmission - delivery of health care - quality of health care technology assessment - biomedical - clinical decision-making - decision-making

Protection of Human and Animal Subjects

This study was reviewed by the Mayo Clinic Institutional Review Board and deemed “exempt.”




Publication History

Received: 29 April 2020

Accepted: 17 July 2020

Article published online:
02 September 2020

Georg Thieme Verlag KG
Stuttgart · New York

 

  • References

  • 1 Car J, Sheikh A, Wicks P, Williams MS. Beyond the hype of big data and artificial intelligence: building foundations for knowledge and wisdom. BMC Med 2019; 17 (01) 143
    CrossrefPubMedSearch in Google Scholar
  • 2 Jamei M, Nisnevich A, Wetchler E, Sudat S, Liu E. Predicting all-cause risk of 30-day hospital readmission using artificial neural networks. PLoS One 2017; 12 (07) e0181173
    Search in Google Scholar
  • 3 Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med 2018; 24 (11) 1716-1720
    CrossrefPubMedSearch in Google Scholar
  • 4 Seymour CW, Kennedy JN, Wang S. , et al. Derivation, validation, and potential treatment implications of novel clinical phenotypes for sepsis. JAMA 2019; 321 (20) 2003-2017
    CrossrefPubMedSearch in Google Scholar
  • 5 Hasan M. Readmission of patients to hospital: still ill defined and poorly understood. Int J Qual Health Care 2001; 13 (03) 177-179
    CrossrefPubMedSearch in Google Scholar
  • 6 Clarke A. Readmission to hospital: a measure of quality or outcome?. Qual Saf Health Care 2004; 13 (01) 10-11
    CrossrefPubMedSearch in Google Scholar
  • 7 Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 2009; 360 (14) 1418-1428
    CrossrefPubMedSearch in Google Scholar
  • 8 Kripalani S, Theobald CN, Anctil B, Vasilevskis EE. Reducing hospital readmission rates: current strategies and future directions. Annu Rev Med 2014; 65: 471-485
    CrossrefPubMedSearch in Google Scholar
  • 9 Desai NR, Ross JS, Kwon JY. , et al. Association between hospital penalty status under the hospital readmission reduction program and readmission rates for target and nontarget conditions. JAMA 2016; 316 (24) 2647-2656
    CrossrefPubMedSearch in Google Scholar
  • 10 Lee EW. Selecting the best prediction model for readmission. J Prev Med Public Health 2012; 45 (04) 259-266
    CrossrefPubMedSearch in Google Scholar
  • 11 Cholleti S, Post A, Gao J. , et al. Leveraging derived data elements in data analytic models for understanding and predicting hospital readmissions. AMIA Annu Symp Proc 2012; 2012: 103-111
    PubMedSearch in Google Scholar
  • 12 Kulkarni P, Smith LD, Woeltje KF. Assessing risk of hospital readmissions for improving medical practice. Health Care Manage Sci 2016; 19 (03) 291-299
    CrossrefPubMedSearch in Google Scholar
  • 13 Swain MJ, Kharrazi H. Feasibility of 30-day hospital readmission prediction modeling based on health information exchange data. Int J Med Inform 2015; 84 (12) 1048-1056
    CrossrefPubMedSearch in Google Scholar
  • 14 Artetxe A, Beristain A, Graña M. Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Programs Biomed 2018; 164: 49-64
    CrossrefPubMedSearch in Google Scholar
  • 15 van Walraven C, Dhalla IA, Bell C. , et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ 2010; 182 (06) 551-557
    CrossrefPubMedSearch in Google Scholar
  • 16 Cotter PE, Bhalla VK, Wallis SJ, Biram RW. Predicting readmissions: poor performance of the LACE index in an older UK population. Age Ageing 2012; 41 (06) 784-789
    CrossrefPubMedSearch in Google Scholar
  • 17 Damery S, Combes G. Evaluating the predictive strength of the LACE index in identifying patients at high risk of hospital readmission following an inpatient episode: a retrospective cohort study. BMJ Open 2017; 7 (07) e016921
    CrossrefPubMedSearch in Google Scholar
  • 18 Hopkins BS, Yamaguchi JT, Garcia R. , et al. Using machine learning to predict 30-day readmissions after posterior lumbar fusion: an NSQIP study involving 23,264 patients. J Neurosurg Spine 2019; 1-8
    PubMedSearch in Google Scholar
  • 19 Mahajan SM, Ghani R. Using ensemble machine learning methods for predicting risk of readmission for heart failure. Stud Health Technol Inform 2019; 264: 243-247
    PubMedSearch in Google Scholar
  • 20 Wolff P, Graña M, Ríos SA, Yarza MB. Machine learning readmission risk modeling: a pediatric case study. BioMed Res Int 2019; 2019: 8532892
    CrossrefPubMedSearch in Google Scholar
  • 21 Eckert C, Nieves-Robbins N, Spieker E. , et al. Development and prospective validation of a machine learning-based risk of readmission model in a large military hospital. Appl Clin Inform 2019; 10 (02) 316-325
    Thieme ConnectPubMedSearch in Google Scholar
  • 22 Awan SE, Bennamoun M, Sohel F, Sanfilippo FM, Dwivedi G. Machine learning-based prediction of heart failure readmission or death: implications of choosing the right model and the right metrics. ESC Heart Fail 2019; 6 (02) 428-435
    CrossrefPubMedSearch in Google Scholar
  • 23 Min X, Yu B, Wang F. Predictive modeling of the hospital readmission risk from patients' claims data using machine learning: a case study on COPD. Sci Rep 2019; 9 (01) 2362
    CrossrefPubMedSearch in Google Scholar
  • 24 Kalagara S, Eltorai AEM, Durand WM, DePasse JM, Daniels AH. Machine learning modeling for predicting hospital readmission following lumbar laminectomy. J Neurosurg Spine 2018; 30 (03) 344-352
    CrossrefPubMedSearch in Google Scholar
  • 25 Kakarmath S, Golas S, Felsted J, Kvedar J, Jethwani K, Agboola S. Validating a machine learning algorithm to predict 30-day re-admissions in patients with heart failure: protocol for a prospective cohort study. JMIR Res Protoc 2018; 7 (09) e176
    CrossrefPubMedSearch in Google Scholar
  • 26 Mahajan SM, Mahajan AS, King R, Negahban S. Predicting risk of 30-day readmissions using two emerging machine learning methods. Stud Health Technol Inform 2018; 250: 250-255
    PubMedSearch in Google Scholar
  • 27 Baig MM, Hua N, Zhang E. , et al. A machine learning model for predicting risk of hospital readmission within 30 days of discharge: validated with LACE index and patient at risk of hospital readmission (PARR) model. Med Biol Eng Comput 2020; 58 (07) 1459-1466
    CrossrefPubMedSearch in Google Scholar
  • 28 Gupta S, Ko DT, Azizi P. , et al. Evaluation of machine learning algorithms for predicting readmission after acute myocardial infarction using routinely collected clinical data. Can J Cardiol 2020; 36 (06) 878-885
    CrossrefPubMedSearch in Google Scholar
  • 29 Baig MM, Hua N, Zhang E. , et al. Machine learning-based risk of hospital readmissions: predicting acute readmissions within 30 days of discharge. Conf Proc IEEE Eng Med Biol Soc 2019; 2019: 2178-2181
    PubMedSearch in Google Scholar
  • 30 Goto T, Jo T, Matsui H, Fushimi K, Hayashi H, Yasunaga H. Machine learning-based prediction models for 30-day readmission after hospitalization for chronic obstructive pulmonary disease. COPD 2019; 16 (5-6): 338-343
    CrossrefPubMedSearch in Google Scholar
  • 31 Awan SE, Bennamoun M, Sohel F, Sanfilippo FM, Chow BJ, Dwivedi G. Feature selection and transformation by machine learning reduce variable numbers and improve prediction for heart failure readmission or death. PLoS One 2019; 14 (06) e0218760
    CrossrefPubMedSearch in Google Scholar
  • 32 Morgan DJ, Bame B, Zimand P. , et al. Assessment of machine learning vs standard prediction rules for predicting hospital readmissions. JAMA Netw Open 2019; 2 (03) e190348
    CrossrefPubMedSearch in Google Scholar
  • 33 Golas SB, Shibahara T, Agboola S. , et al. A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak 2018; 18 (01) 44
    CrossrefPubMedSearch in Google Scholar
  • 34 Rojas JC, Carey KA, Edelson DP, Venable LR, Howell MD, Churpek MM. Predicting intensive care unit readmission with machine learning using electronic health record data. Ann Am Thorac Soc 2018; 15 (07) 846-853
    CrossrefPubMedSearch in Google Scholar
  • 35 Desautels T, Das R, Calvert J. , et al. Prediction of early unplanned intensive care unit readmission in a UK tertiary care hospital: a cross-sectional machine learning approach. BMJ Open 2017; 7 (09) e017199
    CrossrefPubMedSearch in Google Scholar
  • 36 Garcia-Arce A, Rico F, Zayas-Castro JL. Comparison of machine learning algorithms for the prediction of preventable hospital readmissions. J Healthc Qual 2018; 40 (03) 129-138
    CrossrefPubMedSearch in Google Scholar
  • 37 Mortazavi BJ, Downing NS, Bucholz EM. , et al. Analysis of machine learning techniques for heart failure readmissions. Circ Cardiovasc Qual Outcomes 2016; 9 (06) 629-640
    CrossrefPubMedSearch in Google Scholar
  • 38 Shameer K, Johnson KW, Yahi A. , et al. Predictive modeling of hospital readmission rates using electronic medical record-wide machine learning: a case-study using Mount Sinai Heart Failure Cohort. Pac Symp Biocomput 2017; 22: 276-287
    Search in Google Scholar
  • 39 Frizzell JD, Liang L, Schulte PJ. , et al. Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches. JAMA Cardiol 2017; 2 (02) 204-209
    CrossrefPubMedSearch in Google Scholar
  • 40 Hood CM, Gennuso KP, Swain GR, Catlin BB. County health rankings: relationships between determinant factors and health outcomes. Am J Prev Med 2016; 50 (02) 129-135
    CrossrefPubMedSearch in Google Scholar
  • 41 Negarandeh R, Zolfaghari M, Bashi N, Kiarsi M. Evaluating the effect of monitoring through telephone (tele-monitoring) on self-care behaviors and readmission of patients with heart failure after discharge. Appl Clin Inform 2019; 10 (02) 261-268
    Thieme ConnectPubMedSearch in Google Scholar
  • 42 Institute of Medicine. Evidence-Based Medicine and the Changing Nature of Healthcare: 2007 IOM Annual Meeting Summary. Washington (DC): National Academies Press; 2008
    Search in Google Scholar
  • 43 Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood) 2008; 27 (03) 759-769
    CrossrefPubMedSearch in Google Scholar
  • 44 Chen S, Bergman D, Miller K, Kavanagh A, Frownfelter J, Showalter J. Using applied machine learning to predict healthcare utilization based on socioeconomic determinants of care. Am J Manag Care 2020; 26 (01) 26-31
    CrossrefPubMedSearch in Google Scholar
  • 45 Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA 2018; 320 (21) 2199-2200
    CrossrefPubMedSearch in Google Scholar
  • 46 Navathe AS, Zhong F, Lei VJ. , et al. Hospital readmission and social risk factors identified from physician notes. Health Serv Res 2018; 53 (02) 1110-1136
    CrossrefPubMedSearch in Google Scholar
  • 47 Romero-Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care 2015; 19: 285
    CrossrefPubMedSearch in Google Scholar
  • 48 Yu S, Farooq F, van Esbroeck A, Fung G, Anand V, Krishnapuram B. Predicting readmission risk with institution-specific prediction models. Artif Intell Med 2015; 65 (02) 89-96
    CrossrefPubMedSearch in Google Scholar
  • 49 Lazer D, Kennedy R, King G, Vespignani A. Big data. The parable of Google Flu: traps in big data analysis. Science 2014; 343 (6176): 1203-1205
    CrossrefPubMedSearch in Google Scholar