Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2022; 13(02): 339-354
DOI: 10.1055/s-0042-1743243
Review Article

Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Han Chang Lim   *
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
2   Department of Health, eHealth Queensland, Queensland Government, Brisbane, Australia
,
Jodie A. Austin   *
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
2   Department of Health, eHealth Queensland, Queensland Government, Brisbane, Australia
,
Anton H. van der Vegt   *
3   Information Engineering Lab, School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Brisbane, Australia
,
Amir Kamel Rahimi
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
4   Digital Health Cooperative Research Centre, Australian Government, Sydney, New South Wales, Australia
,
Oliver J. Canfell
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
4   Digital Health Cooperative Research Centre, Australian Government, Sydney, New South Wales, Australia
5   UQ Business School, Faculty of Business, Economics and Law, The University of Queensland, St. Lucia, Brisbane, Australia
,
Jayden Mifsud
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
,
Jason D. Pole
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
,
Michael A. Barras
6   School of Pharmacy, Faculty of Health and Behavioural Sciences, The University of Queensland, PACE Precinct, Woolloongabba, Brisbane, Australia
7   Pharmacy Department, Princess Alexandra Hospital, Woolloongabba, Brisbane, Australia
,
Tobias Hodgson
5   UQ Business School, Faculty of Business, Economics and Law, The University of Queensland, St. Lucia, Brisbane, Australia
,
Sally Shrapnel
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
8   School of Mathematics and Physics, Faculty of Science, The University of Queensland, St Lucia, Brisbane, Australia
,
Clair M. Sullivan
1   Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
9   Department of Health, Metro North Hospital and Health Service, Queensland Government, Herston QLD, Australia
› Author Affiliations Funding This study was funded by the Digital Health CRC, grant no.: STARS 0034.
› Further Information
Also available at
   Permissions and Reprints

Abstract

Objective A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.

Methods Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.

Results A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.

Conclusion Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.

Keywords

learning health care system - electronic health records and systems - clinical decision support - hospital information systems - clinical data management - dashboard - digital hospital

Author Contributions

Conception and design: H.C.L. and C.M.S. Data collection: J.D.P. and C.M.S. Data extraction: H.C.L., A.K.R., J.M., and A.V.D.V. Quality Assessment: J.A.A. and H.C.L. Data analysis and interpretation: H.C.L., J.A.A., A.V.D.V., and C.M.S. Drafting the manuscript: H.C.L., J.A.A., A.V.D.V., and C.M.S. This article has co–first authorship by three authors. H.C.L., J.A.A., and A.V.D.V. contributed equally and have the right to their name first in their CV. Critical revision of article: H.C.L., J.A.A., A.V.D.V., A.K.R., J.M., O.J.C., J.D.P., M.A.B., T.H., S.S., and C.M.S. All authors contributed to the article and approved the submitted version.


Protection of Human and Animal Subjects

Human and/or animal subjects were not involved in completing the present review.


* Marked Authors Are Co–First Authors.


Supplementary Material



Publication History

Received: 29 July 2021

Accepted: 09 January 2022

Article published online:
06 April 2022

© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Charles D, Gabriel M, Furukawa MF. Adoption of electronic health record systems among U. S. non -federal acute care hospitals: 2008–2015. Accessed October 7, 2021 at: https://www.healthit.gov/sites/default/files/briefs/2015_hospital_adoption_db_v17.pdf
    PubMedGoogle Scholar
  • 2 Carroll JS, Quijada MA. Redirecting traditional professional values to support safety: changing organisational culture in health care. Qual Saf Health Care 2004; 13 (Suppl. 02) ii16-ii21
    PubMedGoogle Scholar
  • 3 Aggarwal A, Aeran H, Rathee M. Quality management in healthcare: The pivotal desideratum. J Oral Biol Craniofac Res 2019; 9 (02) 180-182
    CrossrefPubMedGoogle Scholar
  • 4 Oaten J, Stayner G, Ballard J. et al. Baby deaths: hospital failures: an independent investigation has found a series of failures may have contributed to the deaths of 7 babies at a regional Victorian hospital. Accessed July 21, 2021 at: https://search-informit-org.ezproxy.library.uq.edu.au/doi/10.3316/tvnews.tsm201510160050
    PubMedGoogle Scholar
  • 5 Barnett A, Winning M, Canaris S, Cleary M, Staib A, Sullivan C. Digital transformation of hospital quality and safety: real-time data for real-time action. Aust Health Rev 2019; 43 (06) 656-661
    CrossrefPubMedGoogle Scholar
  • 6 Sullivan C, Staib A, McNeil K, Rosengren D, Johnson I. Queensland digital health clinical charter: a clinical consensus statement on priorities for digital health in hospitals. Aust Health Rev 2020; 44 (05) 661-665
    CrossrefPubMedGoogle Scholar
  • 7 Mandl KD, Kohane IS, McFadden D. et al. Scalable Collaborative Infrastructure for a Learning Healthcare System (SCILHS): architecture. J Am Med Inform Assoc 2014; 21 (04) 615-620
    CrossrefPubMedGoogle Scholar
  • 8 Olsen LA, Aisner D, McGinnis JM. eds; Institute of Medicine (US) Roundtable on Evidence-Based Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press (US); 2007
    Google Scholar
  • 9 Platt JE, Raj M, Wienroth M. An analysis of the learning health system in its first decade in practice: scoping review. J Med Internet Res 2020; 22 (03) e17026
    CrossrefPubMedGoogle Scholar
  • 10 Li X, Zhao X, Pu W, Chen P, Liu F, He Z. Optimal decisions for operations management of BDAR: a military industrial logistics data analytics perspective. Comput Ind Eng 2019; 137: 106100
    CrossrefPubMedGoogle Scholar
  • 11 Sullivan C, Staib A, Khanna S. et al. The National Emergency Access Target (NEAT) and the 4-hour rule: time to review the target. Med J Aust 2016; 204 (09) 354
    CrossrefPubMedGoogle Scholar
  • 12 Auliya RAknuranda I, Tolle H. A systematic literature review on healthcare dashboards development: trends, issues, methods, and frameworks. Adv Sci Lett 2018; 24 (11) 8632-8639
    CrossrefPubMedGoogle Scholar
  • 13 Buttigieg SC, Pace A, Rathert C. Hospital performance dashboards: a literature review. J Health Organ Manag 2017; 31 (03) 385-406
    CrossrefPubMedGoogle Scholar
  • 14 Dowding D, Randell R, Gardner P. et al. Dashboards for improving patient care: review of the literature. Int J Med Inform 2015; 84 (02) 87-100
    Google Scholar
  • 15 Khairat SSDA, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The impact of visualization dashboards on quality of care and clinician satisfaction: integrative literature review. JMIR Human Factors 2018; 5 (02) e22
    CrossrefPubMedGoogle Scholar
  • 16 Maktoobi SMelchiori M. A brief survey of recent clinical dashboards. Accessed January 25, 2022 at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1073.1848&rep=rep1&type=pdf
    PubMedGoogle Scholar
  • 17 West VL, Borland D, Hammond WE. Innovative information visualization of electronic health record data: a systematic review. J Am Med Inform Assoc 2015; 22 (02) 330-339
    CrossrefPubMedGoogle Scholar
  • 18 Wilbanks BALP, Langford PA. A review of dashboards for data analytics in nursing. Comput Inform Nurs 2014; 32 (11) 545-549
    CrossrefPubMedGoogle Scholar
  • 19 Veritas Health Innovation. Covidence: systematic review. Accessed January 25, 2022 at: https://unimelb.libguides.com/sysrev/covidence
    PubMedGoogle Scholar
  • 20 Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract 2012; 18 (04) 746-752
    CrossrefPubMedGoogle Scholar
  • 21 Bersani K, Fuller TE, Garabedian P. et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
    Article in Thieme ConnectPubMedGoogle Scholar
  • 22 Cox ZLP, Lewis CMNPCC, Lai P, Lenihan DJMD. Validation of an automated electronic algorithm and “dashboard” to identify and characterize decompensated heart failure admissions across a medical center. Am Heart J 2017; 183: 40-48
    CrossrefPubMedGoogle Scholar
  • 23 Fletcher GS, Aaronson BA, White AA, Julka R. Effect of a real-time electronic dashboard on a rapid response system. J Med Syst 2017; 42 (01) 5
    CrossrefPubMedGoogle Scholar
  • 24 Fuller TE, Garabedian PM, Lemonias DP. et al. Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management. Appl Ergon 2020; 85: 103047
    CrossrefPubMedGoogle Scholar
  • 25 Mlaver E, Schnipper JL, Boxer RB. et al. User-centered collaborative design and development of an inpatient safety dashboard. Jt Comm J Qual Patient Saf 2017; 43 (12) 676-685
    PubMedGoogle Scholar
  • 26 Paulson SS, Dummett BA, Green J, Scruth E, Reyes V, Escobar GJ. What do we do after the pilot is done? Implementation of a hospital early warning system at scale. Jt Comm J Qual Patient Saf 2020; 46 (04) 207-216
    PubMedGoogle Scholar
  • 27 Schall Jr MC, Cullen L, Pennathur P, Chen H, Burrell K, Matthews G. Usability evaluation and implementation of a health information technology dashboard of Evidence-Based Quality Indicators. Comput Inform Nurs 2017; 35 (06) 281-288
    CrossrefPubMedGoogle Scholar
  • 28 Ye C, Wang O, Liu M. et al. A real-time early warning system for monitoring inpatient mortality risk: prospective study using electronic medical record data. J Med Internet Res 2019; 21 (07) e13719
    CrossrefPubMedGoogle Scholar
  • 29 Franklin A, Gantela S, Shifarraw S. et al. Dashboard visualizations: supporting real-time throughput decision-making. J Biomed Inform 2017; 71: 211-221
    Google Scholar
  • 30 Ibrahim H, Sorrell S, Nair SC, Al Romaithi A, Al Mazrouei S, Kamour A. Rapid development and utilization of a clinical intelligence dashboard for frontline clinicians to optimize critical resources during Covid-19. Acta Inform Med 2020; 28 (03) 209-213
    CrossrefPubMedGoogle Scholar
  • 31 Kurtzman G, Dine J, Epstein A. et al. Internal medicine resident engagement with a laboratory utilization dashboard: mixed methods study. J Hosp Med 2017; 12 (09) 743-746
    CrossrefPubMedGoogle Scholar
  • 32 Merkel MJ, Edwards R, Ness J. et al. statewide real-time tracking of beds and ventilators during coronavirus disease 2019 and beyond. Crit Care Explor 2020; 2 (06) e0142
    CrossrefPubMedGoogle Scholar
  • 33 Staib A, Sullivan C, Jones M, Griffin B, Bell A, Scott I. The ED-inpatient dashboard: Uniting emergency and inpatient clinicians to improve the efficiency and quality of care for patients requiring emergency admission to hospital. Emerg Med Australas 2017; 29 (03) 363-366
    CrossrefPubMedGoogle Scholar
  • 34 Yoo J, Jung KY, Kim T. et al. A real-time autonomous dashboard for the emergency department: 5-year case study. JMIR Mhealth Uhealth 2018; 6 (11) e10666
    CrossrefPubMedGoogle Scholar
  • 35 Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011; 30 (12) 2310-2317
    CrossrefPubMedGoogle Scholar
  • 36 Health IT. gov. 2020–2025 federal health it strategic plan 2020. Accessed October 5, 2021 at: https://www.healthit.gov/sites/default/files/page/2020-10/Federal%20Health%20IT%20Strategic%20Plan_2020_2025.pdf
    PubMedGoogle Scholar
  • 37 Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci 2016; 11 (01) 33
    CrossrefPubMedGoogle Scholar
  • 38 CFIR Research Team-Center for Clinical Management Research. Consolidated framework for implementation research. Accessed July 2, 2021 at: https://cfirguide.org/
    PubMedGoogle Scholar
  • 39 Hunter SC, Kim B, Mudge A. et al. Experiences of using the i-PARIHS framework: a co-designed case study of four multi-site implementation projects. BMC Health Serv Res 2020; 20 (01) 573
    CrossrefPubMedGoogle Scholar
  • 40 Safaeinili N, Brown-Johnson C, Shaw JG, Mahoney M, Winget M. CFIR simplified: pragmatic application of and adaptations to the Consolidated Framework for Implementation Research (CFIR) for evaluation of a patient-centered care transformation within a learning health system. Learn Health Syst 2019; 4 (01) e10201
    CrossrefPubMedGoogle Scholar
  • 41 Waitman LR, Phillips IE, McCoy AB. et al. Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy. Jt Comm J Qual Patient Saf 2011; 37 (07) 326-332
    PubMedGoogle Scholar
  • 42 Moorman LP. Principles for real-world implementation of bedside predictive analytics monitoring. Appl Clin Inform 2021; 12 (04) 888-896
    Article in Thieme ConnectPubMedGoogle Scholar