Download PDF
CC BY 4.0 · ACI open 2024; 08(01): e33-e42
DOI: 10.1055/s-0044-1782604
Research Article

A Usability Survey of a Quality Improvement Data Visualization Tool among Medical Intensive Care Unit Nurses

Abigail M. Williams   *
1   University of Virginia School of Medicine, Charlottesville, Virginia, United States
,
Claire L. Davis   *
2   Division of Pulmonary and Critical Care Medicine, Department of Medicine, University of Virginia Health System, Charlottesville, Virginia, United States
,
Margot Bjoring
3   Department of Quality and Performance Improvement, University of Virginia Health System, Charlottesville, Virginia, United States
,
Kris Blackstone
4   Department of Nursing, University of Virginia Health System, Charlottesville, Virginia, United States
,
Andrew J. Barros
2   Division of Pulmonary and Critical Care Medicine, Department of Medicine, University of Virginia Health System, Charlottesville, Virginia, United States
,
Kyle B. Enfield
2   Division of Pulmonary and Critical Care Medicine, Department of Medicine, University of Virginia Health System, Charlottesville, Virginia, United States
› Author Affiliations Funding A.J.B. is an iTHRIV Scholar. The iTHRIV Scholars Program is supported in part by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award numbers UL1TR003015 and KL2TR003016.
› Further Information
   Permissions and Reprints

Abstract

Background Cognitive overload is prevalent among intensive care unit (ICU) clinicians. Data visualization may decrease cognitive load by assisting with data interpretation and task prioritization. We developed the Bundle Board to display real-time data from the electronic medical record (EMR), highlighting opportunities for action in standardized ICU patient care. This study evaluates the practical usability of this data visualization tool among nurses in the ICU.

Methods The tool is offered as an application separate from the EMR and was available in the medical ICU for 8 months before we surveyed unit nursing staff. To evaluate usability of the tool, we adapted the Health-Information Technology Usability Scale and included an option to provide open-ended feedback. Survey data were analyzed using quantitative and qualitative methods.

Results ICU nurses were invited to participate through email and verbal announcements. Of the potential participants, 38% (N = 47) responded. The survey demonstrated that the tool was perceived as usable. For each subscale, mean scores were as follows: Perceived Ease of Use 4.40, Impact 4.14, User Control 4.07, and Perceived Usefulness 3.61. There were no significant differences between core and contracted nurses or after stratifying by duration of Bundle Board use. Fifteen respondents completed the optional free-text portion of the survey. Qualitative analysis revealed six subthemes focusing on perceived impacts on quality and safety, cognitive burden and workload, and emotional impact of the Bundle Board.

Conclusion The Bundle Board demonstrated good usability among ICU nurses, who provided substantive feedback for its improvement. These observations may be generalizable to other comparable interventions. Iterative feedback from end users is vital to developing and implementing a digital health intervention. Our study provides a framework for performing a usability analysis within a specific clinician population and environment.

Keywords

data visualization - intensive and critical care - interfaces and usability - dashboard - quality

* Co-first authors.


Supplementary Material



Publication History

Received: 24 May 2023

Accepted: 30 January 2024

Article published online:
05 April 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Pickering BW, Herasevich V, Ahmed A, Gajic O. Novel representation of clinical information in the ICU: developing user interfaces which reduce information overload. Appl Clin Inform 2010; 1 (02) 116-131
    Article in Thieme ConnectPubMedGoogle Scholar
  • 2 Patel V, Buchman T. Cognitive overload in the ICU. Patient Safety Network. August 21, 2016. Accessed December 20, 2023 at: https://psnet.ahrq.gov/web-mm/cognitive-overload-icu
    PubMedGoogle Scholar
  • 3 Moacdieh N, Sarter N. Clutter in electronic medical records: examining its performance and attentional costs using eye tracking. Hum Factors 2015; 57 (04) 591-606
    CrossrefPubMedGoogle Scholar
  • 4 Kutney-Lee A, Brooks Carthon M, Sloane DM, Bowles KH, McHugh MD, Aiken LH. Electronic health record usability: associations with nurse and patient outcomes in hospitals. Med Care 2021; 59 (07) 625-631
    CrossrefPubMedGoogle Scholar
  • 5 Moy AJ, Hobensack M, Marshall K. et al. Understanding the perceived role of electronic health records and workflow fragmentation on clinician documentation burden in emergency departments. J Am Med Inform Assoc 2023; 30 (05) 797-808
    CrossrefPubMedGoogle Scholar
  • 6 Zhao JY, Kessler EG, Guo WA. Interprofessional communication goes up when the electronic health record goes down. J Surg Educ 2019; 76 (02) 512-518
    CrossrefPubMedGoogle Scholar
  • 7 Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci 2015; 10: 119
    CrossrefPubMedGoogle Scholar
  • 8 Ely EW. The ABCDEF bundle: science and philosophy of how ICU liberation serves patients and families. Crit Care Med 2017; 45 (02) 321-330
    CrossrefPubMedGoogle Scholar
  • 9 Hales B, Terblanche M, Fowler R, Sibbald W. Development of medical checklists for improved quality of patient care. Int J Qual Health Care 2008; 20 (01) 22-30
    CrossrefPubMedGoogle Scholar
  • 10 Fuller TE, Garabedian PM, Lemonias DP. et al. Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management. Appl Ergon 2020; 85: 103047
    CrossrefPubMedGoogle Scholar
  • 11 Khairat SS, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The Impact of visualization dashboards on quality of care and clinician satisfaction: integrative literature review. JMIR Human Factors 2018; 5 (02) e22
    CrossrefPubMedGoogle Scholar
  • 12 Lin YL, Trbovich P, Kolodzey L, Nickel C, Guerguerian AM. Association of data integration technologies with intensive care clinician performance: a systematic review and meta-analysis. JAMA Netw Open 2019; 2 (05) e194392
    CrossrefPubMedGoogle Scholar
  • 13 Chemparathy A, Seneviratne MG, Ward A. et al. Development and implementation of a real-time bundle-adherence dashboard for central line-associated bloodstream infections. Pediatr Qual Saf 2021; 6 (04) e431
    CrossrefPubMedGoogle Scholar
  • 14 Davis CL, Bjoring M, Hursh J. et al. The ICU bundle board: a novel real-time data visualization tool to improve maintenance care for invasive catheters. Appl Clin Inform 2023; 14 (05) 892-902
    Article in Thieme ConnectPubMedGoogle Scholar
  • 15 Pageler NM, Longhurst CA, Wood M. et al. Use of electronic medical record-enhanced checklist and electronic dashboard to decrease CLABSIs. Pediatrics 2014; 133 (03) e738-e746
    CrossrefPubMedGoogle Scholar
  • 16 Shaw SJ, Jacobs B, Stockwell DC, Futterman C, Spaeder MC. Effect of a real-time pediatric ICU safety bundle dashboard on quality improvement measures. Jt Comm J Qual Patient Saf 2015; 41 (09) 414-420
    PubMedGoogle Scholar
  • 17 Lai CH, Li KW, Hu FW. et al. Integration of an intensive care unit visualization dashboard (i-Dashboard) as a platform to facilitate multidisciplinary rounds: cluster-randomized controlled trial. J Med Internet Res 2022; 24 (05) e35981
    CrossrefPubMedGoogle Scholar
  • 18 Khasnabish S, Burns Z, Couch M, Mullin M, Newmark R, Dykes PC. Best practices for data visualization: creating and evaluating a report for an evidence-based fall prevention program. J Am Med Inform Assoc 2020; 27 (02) 308-314
    CrossrefPubMedGoogle Scholar
  • 19 Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19 (03) 413-422
    CrossrefPubMedGoogle Scholar
  • 20 Liu K, Nakamura K, Katsukawa H. et al. Implementation of the ABCDEF bundle for critically ill ICU patients during the COVID-19 pandemic: a multi-national 1-day point prevalence study. Front Med (Lausanne) 2021; 8: 735860
    CrossrefPubMedGoogle Scholar
  • 21 International Organization of Standardization. Ergonomics of Human-System Interaction—Part 11: Usability: Definitions and Concepts; 2018. Accessed January 1, 2023 at: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en
    PubMedGoogle Scholar
  • 22 Vincent JL, Moreno R, Takala J. et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996; 22 (07) 707-710
    CrossrefPubMedGoogle Scholar
  • 23 Yen PY, Wantland D, Bakken S. Development of a customizable Health IT Usability Evaluation Scale. AMIA Annu Symp Proc 2010; 2010: 917-921
    PubMedGoogle Scholar
  • 24 Yen PY, Sousa KH, Bakken S. Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results. J Am Med Inform Assoc 2014; 21 (e2): e241-e248
    CrossrefPubMedGoogle Scholar
  • 25 Bersani K, Fuller TE, Garabedian P. et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
    Article in Thieme ConnectPubMedGoogle Scholar
  • 26 Mlaver E, Schnipper JL, Boxer RB. et al. User-centered collaborative design and development of an inpatient safety dashboard. Jt Comm J Qual Patient Saf 2017; 43 (12) 676-685
    PubMedGoogle Scholar
  • 27 R Studio Team. R Studio. 2022 . Accessed March 1, 2023 at: http://www.rstudio.com/
    PubMedGoogle Scholar
  • 28 Dedoose. 2023 . March 1, 2023 at: www.dedoose.com
    PubMedGoogle Scholar
  • 29 Hennink M, Hutter I, Bailey A. Qualitative Research Methods. 1st ed.. Sage; 2010
    Google Scholar
  • 30 Schall Jr MC, Cullen L, Pennathur P, Chen H, Burrell K, Matthews G. Usability evaluation and implementation of a health information technology dashboard of evidence-based quality indicators. Comput Inform Nurs 2017; 35 (06) 281-288
    CrossrefPubMedGoogle Scholar
  • 31 Effken JA, Loeb RG, Kang Y, Lin ZC. Clinical information displays to improve ICU outcomes. Int J Med Inform 2008; 77 (11) 765-777
    CrossrefPubMedGoogle Scholar
  • 32 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; 19 (Suppl 3, Suppl 3): i68-i74
    CrossrefPubMedGoogle Scholar
  • 33 Wisner K, Lyndon A, Chesla CA. The electronic health record's impact on nurses' cognitive work: an integrative review. Int J Nurs Stud 2019; 94: 74-84
    CrossrefPubMedGoogle Scholar
  • 34 Coblis—Color Blindness Simulator – Colblindor. Accessed September 1, 2023 at: https://www.color-blindness.com/coblis-color-blindness-simulator/
    PubMedGoogle Scholar
  • 35 Meyer VM, Benjamens S, Moumni ME, Lange JFM, Pol RA. Global overview of response rates in patient and health care professional surveys in surgery: a systematic review. Ann Surg 2022; 275 (01) e75-e81
    CrossrefPubMedGoogle Scholar
  • 36 Loh KP, Liu J, Ganzhorn S, Sanabria G, Schnall R. Establishing a usability cut-point for the health information technology usability evaluation scale (Health-ITUES). Int J Med Inform 2022; 160: 104713
    CrossrefPubMedGoogle Scholar
  • 37 Elias J, Moy A, Lucas E. et al. Assessing clinical staff usability & satisfaction with documentation & information retrieval prior to an electronic health record implementation. Podium Presentation presented at: AMIA Annual Symposium; 2020 ; Virtual. Accessed May 6, 2023 at: https://knowledge.amia.org/72332-amia-1.4602255/t004-1.4605866/t004-1.4605867/3414558-1.4606120/3414558-1.4606121?timeStamp=1683400189035#tab_0=1
    PubMedGoogle Scholar
  • 38 Campbell CM, Prapanjaroensin A, Anusiewicz CV, Baernholdt M, Jones T, Patrician PA. Variables associated with missed nursing care in Alabama: a cross-sectional analysis. J Nurs Manag 2020; 28 (08) 2174-2184
    CrossrefPubMedGoogle Scholar
  • 39 Kuniecki M, Pilarczyk J, Wichary S. The color red attracts attention in an emotional context. An ERP study. Front Hum Neurosci 2015; 9: 212
    CrossrefPubMedGoogle Scholar
  • 40 Nielsen J. Usability inspection methods. In: Conference Companion on Human Factors in Computing Systems—CHI '94. ACM Press; 1994: 413-414
    Google Scholar
  • 41 Turner P, Kushniruk A, Nohr C. Are we there yet? human factors knowledge and health information technology - the challenges of implementation and impact. Yearb Med Inform 2017; 26 (01) 84-91
    Article in Thieme ConnectPubMedGoogle Scholar
  • 42 Carayon P, Hoonakker P. Human factors and usability for health information technology: old and new challenges. Yearb Med Inform 2019; 28 (01) 71-77
    Article in Thieme ConnectPubMedGoogle Scholar
  • 43 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
    CrossrefPubMedGoogle Scholar