Unsupervised Machine Learning of Topics Documented by Nurses about Hospitalized Patients Prior to a Rapid-Response Event

Abstract

Background In the hospital setting, it is crucial to identify patients at risk for deterioration before it fully develops, so providers can respond rapidly to reverse the deterioration. Rapid response (RR) activation criteria include a subjective component (“worried about the patient”) that is often documented in nurses' notes and is hard to capture and quantify, hindering active screening for deteriorating patients.

Objectives We used unsupervised machine learning to automatically discover RR event risk/protective factors from unstructured nursing notes.

Methods In this retrospective cohort study, we obtained nursing notes of hospitalized, nonintensive care unit patients, documented from 2015 through 2018 from Partners HealthCare databases. We applied topic modeling to those notes to reveal topics (clusters of associated words) documented by nurses. Two nursing experts named each topic with a representative Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT) concept. We used the concepts along with vital signs and demographics in a time-dependent covariates extended Cox model to identify risk/protective factors for RR event risk.

Results From a total of 776,849 notes of 45,299 patients, we generated 95 stable topics, of which 80 were mapped to 72 distinct SNOMED CT concepts. Compared with a model containing only demographics and vital signs, the latent topics improved the model's predictive ability from a concordance index of 0.657 to 0.720. Thirty topics were found significantly associated with RR event risk at a 0.05 level, and 11 remained significant after Bonferroni correction of the significance level to 6.94E-04, including physical examination (hazard ratio [HR] = 1.07, 95% confidence interval [CI], 1.03–1.12), informing doctor (HR = 1.05, 95% CI, 1.03–1.08), and seizure precautions (HR = 1.08, 95% CI, 1.04–1.12).

Conclusion Unsupervised machine learning methods can automatically reveal interpretable and informative signals from free-text and may support early identification of patients at risk for RR events.

Keywords

hospital rapid response team - nursing assessment - electronic health records - survival analysis - nursing notes - natural language processing - machine learning - medicine

Protection of Human and Animal Subjects

The study was approved by the institutional review board of Partners HealthCare System.

• References

• 1 Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapid-response systems as a patient safety strategy: a systematic review. Ann Intern Med 2013; 158 (5 Pt 2): 417-425
• 2 Solomon RS, Corwin GS, Barclay DC, Quddusi SF, Dannenberg MD. Effectiveness of rapid response teams on rates of in-hospital cardiopulmonary arrest and mortality: a systematic review and meta-analysis. J Hosp Med 2016; 11 (06) 438-445
• 3 Huh JW, Lim C-M, Koh Y. , et al. Activation of a medical emergency team using an electronic medical recording-based screening system*. Crit Care Med 2014; 42 (04) 801-808
• 4 Kollef MH, Chen Y, Heard K. , et al. A randomized trial of real-time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med 2014; 9 (07) 424-429
• 5 Collins SA, Cato K, Albers D. , et al. Relationship between nursing documentation and patients' mortality. Am J Crit Care 2013; 22 (04) 306-313
• 6 Bellomo R, Goldsmith D, Uchino S. , et al. Prospective controlled trial of effect of medical emergency team on postoperative morbidity and mortality rates. Crit Care Med 2004; 32 (04) 916-921
• 7 Hillman K, Chen J, Cretikos M. , et al; MERIT study investigators. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet 2005; 365 (9477): 2091-2097
• 8 Parr MJ, Hadfield JH, Flabouris A, Bishop G, Hillman K. The Medical Emergency Team: 12 month analysis of reasons for activation, immediate outcome and not-for-resuscitation orders. Resuscitation 2001; 50 (01) 39-44
• 9 Hodgetts TJ, Kenward G, Vlackonikolis I. , et al. Incidence, location and reasons for avoidable in-hospital cardiac arrest in a district general hospital. Resuscitation 2002; 54 (02) 115-123
• 10 Rubin TN, Chambers A, Smyth P, Steyvers M. Statistical topic models for multi-label document classification. Mach Learn 2012; 88 (01) 157-208
• 11 Wang L, Sha L, Lakin JR. , et al. Development and validation of a deep learning algorithm for mortality prediction in selecting patients with dementia for earlier palliative care interventions. JAMA Netw Open 2019; 2 (07) e196972
• 12 Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res 2003; 3 (Jan): 993-1022
• 13 Perotte A, Ranganath R, Hirsch JS, Blei D, Elhadad N. Risk prediction for chronic kidney disease progression using heterogeneous electronic health record data and time series analysis. J Am Med Inform Assoc 2015; 22 (04) 872-880
• 14 Disease Trajectories and End-of-Life Care for Dementias: Latent Topic Modeling and Trend Analysis Using Clinical Notes. ResearchGate. Available at: https://www.researchgate.net/publication/328899169_Disease_Trajectories_and_End-of-Life_Care_for_Dementias_Latent_Topic_Modeling_and_Trend_Analysis_Using_Clinical_Notes . Accessed December 18, 2018
• 15 Shao Y, Mohanty AF, Ahmed A. , et al. Identification and use of frailty indicators from text to examine associations with clinical outcomes among patients with heart failure. AMIA Annu Symp Proc 2017; 2016: 1110-1118
• 16 Manning CD, Surdeanu M, Bauer J, Finkel J, Bethard SJ, McClosky D. The Stanford CoreNLP Natural Language Processing Toolkit. In: Association for Computational Linguistics (ACL) System Demonstrations; 2014: 55-60 . Available at: http://www.aclweb.org/anthology/P/P14/P14-5010 . Accessed November 25, 2019
• 17 gensim: topic modelling for humans. Available at: https://radimrehurek.com/gensim/ . Accessed March 4, 2018
• 18 Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol 1995; 48 (12) 1495-1501
• 19 Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 1995; 48 (12) 1503-1510
• 20 Andersen PK. Repeated assessment of risk factors in survival analysis. Stat Methods Med Res 1992; 1 (03) 297-315