Download PDF
CC BY 4.0 · ACI open 2024; 08(01): e1-e9
DOI: 10.1055/s-0044-1779698
Research Article

User-centered Design and Formative Evaluation of a Web Application to Collect and Visualize Real-time Clinician Well-being Levels

Derek Shu
1   Department of Biomedical Informatics, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
2   Medical Doctorate Program, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
,
Catherine T. Xu
1   Department of Biomedical Informatics, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
2   Medical Doctorate Program, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
,
Somya Pandey
1   Department of Biomedical Informatics, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
3   Medical Sciences Baccalaureate Program, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
,
Virginia Walls
4   University of Cincinnati Health, Cincinnati, Ohio, United States
,
Kristen Tenney
4   University of Cincinnati Health, Cincinnati, Ohio, United States
5   University of Cincinnati Blush Ash College, Cincinnati, Ohio, United States
,
Abby Georgilis
4   University of Cincinnati Health, Cincinnati, Ohio, United States
,
Lisa Melink
4   University of Cincinnati Health, Cincinnati, Ohio, United States
,
Danny T.Y. Wu
1   Department of Biomedical Informatics, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
3   Medical Sciences Baccalaureate Program, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
,
Jennifer Rose Molano
6   Department of Neurology and Rehabilitation Medicine, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States
› Author Affiliations Funding None.
› Further Information
   Permissions and Reprints

Abstract

Background Clinician burnout is increasingly prevalent in the health care workplace. Hospital leadership needs an informatics tool to measure clinicians' well-being levels and provide empirical evidence to improve their work environment.

Objectives This study aimed to (1) design and implement a web-based application to collect and visualize clinicians' well-being levels and (2) conduct formative usability evaluation.

Methods Clinician and staff well-being champions guided the development of the Well-being Check application. User-centered design and Agile principles were used for incremental development of the app. The app included a customizable survey and an interactive visualization. The survey consisted of six standard, two optional, and three additional questions. The interactive visualization included various charts and word clouds with filters for drill-down analysis. The evaluation was done primarily with the rehabilitation (REHAB) team using data-centered approaches through historical survey data and qualitative coding of the free-text explanations and user-centered approaches through the System Usability Scale (SUS).

Results The evaluation showed that the app appropriately accommodated historical survey data from the REHAB team, enabling the comparison between self-assessed and perceived team well-being levels, and summarized key drivers based on the qualitative coding of the free-text explanations. Responses from the 23 REHAB team members showed an above-average score (SUS: 80.22), indicating high usability of the app.

Conclusion The Well-being Check app was developed in a user-centered manner and evaluated to demonstrate its effectiveness and usability. Future work includes iterative refinement of the app and designing a pre-poststudy using the app to measure the change in clinicians' well-being levels for quality improvement intervention.

Keywords

care teams - interfaces and usability - data visualization - other clinical informatics applications

Author Contributions

The corresponding author (D.T.Y.W.) designed and mentored the first two authors (D.S. and C.T.X.) to conduct the study and analyze the data. The first author (D.S.) drafted the manuscript with the support of the second author (C.T.X.). The third author (S.P.) collected and analyzed the usability scores. The senior author (J.R.M.) provided guidance and resources to support the development activities. All coauthors helped improve the clarity and value of the manuscript by reviewing and revising the manuscript.


Protection of Human and Animal Subjects

No animal subjects were included. The study was considered Not Human Subject Research due to its non-generalizable nature and the use of survey to collect de-identified user feedback.




Publication History

Received: 09 May 2023

Accepted: 02 January 2024

Article published online:
21 February 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 National Academies of Sciences, Engineering, and Medicine, National Academy of Medicine; Committee on Systems Approaches to Improve Patient Care by Supporting Clinician Well-Being. Taking Action Against Clinician Burnout: A Systems Approach to Professional Well-Being. United States: National Academies Press; 2019. . Accessed February 23, 2023 at: http://www.ncbi.nlm.nih.gov/books/NBK552618/
    Google Scholar
  • 2 Chari R, Chang CC, Sauter SL. et al. Expanding the paradigm of occupational safety and health: a new framework for worker well-being. J Occup Environ Med 2018; 60 (07) 589-593
    CrossrefPubMedGoogle Scholar
  • 3 Sullivan EE, Dwiel K, Hunt LS, Conroy K, Gergen Barnett K. Moving the needle on primary care burnout: using a driver diagram to accelerate impact. Healthc (Amst) 2021; 9 (04) 100595
    CrossrefPubMedGoogle Scholar
  • 4 West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med 2018; 283 (06) 516-529
    CrossrefPubMedGoogle Scholar
  • 5 Rotenstein LS, Torre M, Ramos MA. et al. Prevalence of burnout among physicians: a systematic review. JAMA 2018; 320 (11) 1131-1150
    CrossrefPubMedGoogle Scholar
  • 6 Kane L. Medscape US and International Physicians' COVID-19 Experience Report: Risk, Burnout, Loneliness. Published online September 11, 2020. Accessed November 27, 2023
    PubMedGoogle Scholar
  • 7 Prasad K, McLoughlin C, Stillman M. et al. Prevalence and correlates of stress and burnout among U.S. healthcare workers during the COVID-19 pandemic: a national cross-sectional survey study. EClinicalMedicine 2021; 35: 100879
    CrossrefPubMedGoogle Scholar
  • 8 Stehman CR, Testo Z, Gershaw RS, Kellogg AR. Burnout, drop out, suicide: physician loss in Emergency Medicine, Part I. West J Emerg Med 2019; 20 (03) 485-494
    CrossrefPubMedGoogle Scholar
  • 9 Tawfik DS, Profit J, Morgenthaler TI. et al. Physician burnout, well-being, and work unit safety grades in relationship to reported medical errors. Mayo Clin Proc 2018; 93 (11) 1571-1580
    CrossrefPubMedGoogle Scholar
  • 10 Tung YC, Chou YY, Chang YH, Chung KP. Association of intrinsic and extrinsic motivating factors with physician burnout and job satisfaction: a nationwide cross-sectional survey in Taiwan. BMJ Open 2020; 10 (03) e035948
    CrossrefPubMedGoogle Scholar
  • 11 Sliwa JA, Clark GS, Chiodo A. et al. Burnout in diplomates of the American Board of Physical Medicine and Rehabilitation-Prevalence and Potential Drivers: a prospective cross-sectional survey. PM R 2019; 11 (01) 83-89
    CrossrefPubMedGoogle Scholar
  • 12 Rabatin J, Williams E, Baier Manwell L, Schwartz MD, Brown RL, Linzer M. Predictors and outcomes of burnout in primary care physicians. J Prim Care Community Health 2016; 7 (01) 41-43
    CrossrefPubMedGoogle Scholar
  • 13 Aggarwal R, Deutsch JK, Medina J, Kothari N. Resident wellness: an intervention to decrease burnout and increase resiliency and happiness. MedEdPORTAL 2017; 13: 10651
    CrossrefPubMedGoogle Scholar
  • 14 Wu DTY, Xu C, Kim A, Bindhu S, Mah KE, Eckman MH. A scoping review of health information technology in clinician burnout. Appl Clin Inform 2021; 12 (03) 597-620
    Article in Thieme ConnectPubMedGoogle Scholar
  • 15 Tracking Mental Well-Being. Balancing rich sensing and patient needs. Computer 2014; 47 (04) 36-43
    CrossrefPubMedGoogle Scholar
  • 16 Johnson T, Kanjo E. Sensor Fusion and the City: Visualisation and Aggregation of Environmental & Wellbeing Data. Paper presented at: September 7–10, 2021 IEEE International Smart Cities Conference (ISC2). IEEE; 2021: 1-4 DOI: 10.1109/ISC253183.2021.9562852
    CrossrefPubMedGoogle Scholar
  • 17 Chandawarkar A, Chaparro JD. Burnout in clinicians. Curr Probl Pediatr Adolesc Health Care 2021; 51 (11) 101104
    CrossrefPubMedGoogle Scholar
  • 18 usability.gov. User-centered design basics. Accessed November 27, 2023 at: https://www.usability.gov/what-and-why/user-centered-design.html
    PubMedGoogle Scholar
  • 19 Martin RC. Agile Software Development: Principles, Patterns, and Practices. Prentice Hall; 2003
    Google Scholar
  • 20 Watson P. Stress First Aid for Healthcare Workers. National Center for PTSD; . Accessed November 27, 2023 at: https://www.ptsd.va.gov/professional/treat/type/stress_first_aid.asp
    Google Scholar
  • 21 Munzner T. A nested model for visualization design and validation. IEEE Trans Vis Comput Graph 2009; 15 (06) 921-928
    CrossrefPubMedGoogle Scholar
  • 22 Perlo J, Swensen S, Kabcenekk A, Landsman J, Feeley D. IHI Framework for Improving Joy in Work. IHI White Paper. Cambridge, MA: Institute of Healthcare Improvement; . Accessed on February 24, 2023 at: https://www.ncha.org/wp-content/uploads/2018/06/IHIWhitePaper_FrameworkForImprovingJoyInWork.pdf
    Google Scholar
  • 23 Brooke J. SUS - A Quick and Dirty Usability Scale. Accessed November 27, 2023 at: https://hell.meiert.org/core/pdf/sus.pdf
    PubMedGoogle Scholar
  • 24 Nash WP. US Marine Corps and Navy combat and operational stress continuum model: a tool for leaders. Combat Oper Behav Health 2011; 107: 119
    PubMedGoogle Scholar
  • 25 Ervin JN, Kahn JM, Cohen TR, Weingart LR. Teamwork in the intensive care unit. Am Psychol 2018; 73 (04) 468-477
    CrossrefPubMedGoogle Scholar
  • 26 Bruce CR, Miller SM, Zimmerman JL. A qualitative study exploring moral distress in the ICU team: the importance of unit functionality and intrateam dynamics. Crit Care Med 2015; 43 (04) 823-831
    CrossrefPubMedGoogle Scholar
  • 27 Nielsen K, Randall R. The importance of employee participation and perceptions of changes in procedures in a teamworking intervention. Work Stress 2012; 26 (02) 91-111
    CrossrefPubMedGoogle Scholar