The Effect of Electronic Health Record Usability Redesign on Annual Screening Rates in an Ambulatory Setting

Abstract

Objectives Improving the usability of electronic health records (EHR) continues to be a focus of clinicians, vendors, researchers, and regulatory bodies. To understand the impact of usability redesign of an existing, site-configurable feature, we evaluated the user interface (UI) used to screen for depression, alcohol and drug misuse, fall risk, and the existence of advance directive information in ambulatory settings.

Methods As part of a quality improvement project, based on heuristic analysis, the existing UI was redesigned. Using an iterative, user-centered design process, several usability defects were corrected. Summative usability testing was performed as part of the product development and implementation cycle. Clinical quality measures reflecting rolling 12-month rates of screening were examined over 8 months prior to the implementation of the redesigned UI and 9 months after implementation.

Results Summative usability testing demonstrated improvements in task time, error rates, and System Usability Scale scores. Interrupted time series analysis demonstrated significant improvements in all screening rates after implementation of the redesigned UI compared with the original implementation.

Conclusion User-centered redesign of an existing site-specific UI may lead to significant improvements in measures of usability and quality of patient care.

Keywords

electronic health record - interfaces - usability - quality - nurse - ambulatory setting - primary care

Protection of Human and Animal Subjects

The project details were reviewed by the institutional review board who determined the project to be a quality improvement activity and not human subjects research and did not require additional review.

Publication History

Received: 09 April 2020

Accepted: 20 July 2020

Article published online:
09 September 2020

• References

• 1 International Standards Organization. Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems. Vol 9241–210:2019. International Organization for Standardization; 2019 . Available at: https://www.iso.org/standard/77520.html. Accessed July 30, 2020
• 2 Melnick ER, Dyrbye LN, Sinsky CA. , et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
• 3 Committee on Patient Safety and Health Information Technology, Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: National Academies Press (US); 2011
• 4 Howe JL, Adams KT, Hettinger AZ, Ratwani RM. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018; 319 (12) 1276-1278
• 5 Russ AL, Zillich AJ, Melton BL. , et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc 2014; 21 (e2): e287-e296
• 6 Orenstein EW, Boudreaux J, Rollins M. , et al. Formative usability testing reduces severe blood product ordering errors. Appl Clin Inform 2019; 10 (05) 981-990
• 7 Ratwani RM, Savage E, Will A. , et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018; 25 (09) 1197-1201
• 8 Sittig DF, Belmont E, Singh H. Improving the safety of health information technology requires shared responsibility: it is time we all step up. Healthc (Amst) 2018; 6 (01) 7-12
• 9 Siu AL, Bibbins-Domingo K, Grossman DC. , et al; US Preventive Services Task Force (USPSTF). Screening for depression in adults: US Preventive Services Task Force recommendation statement. JAMA 2016; 315 (04) 380-387
• 10 O'Connor EA, Perdue LA, Senger CA. , et al. Screening and behavioral counseling interventions to reduce unhealthy alcohol use in adolescents and adults: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA 2018; 320 (18) 1910-1928
• 11 Grossman DC, Curry SJ, Owens DK. , et al; US Preventive Services Task Force. Interventions to prevent falls in community-dwelling older adults: US Preventive Services Task Force recommendation statement. JAMA 2018; 319 (16) 1696-1704
• 12 Weathers E, O'Caoimh R, Cornally N. , et al. Advance care planning: a systematic review of randomised controlled trials conducted with older adults. Maturitas 2016; 91: 101-109
• 13 Nielsen J. 10 Usability heuristics for user interface design. Nielsen Normal Group, Available at: https://www.nngroup.com/articles/ten-usability-heuristics/ . Published 1994. Accessed May 17, 2020
• 14 Schumacher RM, Lowry SZ. Customized Common Industry Format Template for Electronic Health Record Usability Testing (NISTIR 7742). Washington, DC: National Institute for Standards and Technology (US Department of Commerce); 2010: 1-37
• 15 Sauro J, Dumas JS. Comparison of three one-question, post-task usability questionnaires. Paper presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; April 4–9, 2009; Boston, Massachusetts, United States:
• 16 Brooke J. SUS: a “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL. , eds. Usability Evaluation in Industry. London: Taylor and Francis; 1996: 189-194
• 17 Ellsworth MA, Dziadzko M, O'Horo JC, Farrell AM, Zhang J, Herasevich V. An appraisal of published usability evaluations of electronic health records via systematic review. J Am Med Inform Assoc 2017; 24 (01) 218-226
• 18 Wu DTY, Vennemeyer S, Brown K. , et al. Usability testing of an interactive dashboard for surgical quality improvement in a large congenital heart center. Appl Clin Inform 2019; 10 (05) 859-869
• 19 Abdel-Rahman SM, Gill H, Carpenter SL. , et al. Design and usability of an electronic health record-integrated, point-of-care, clinical decision support tool for modeling and simulation of antihemophilic factors. Appl Clin Inform 2020; 11 (02) 253-264
• 20 Bersani K, Fuller TE, Garabedian P. , et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
• 21 Brown N, Eghdam A, Koch S. Usability evaluation of visual representation formats for emergency department records. Appl Clin Inform 2019; 10 (03) 454-470
• 22 Marcilly R, Schiro J, Beuscart-Zéphir MC, Magrabi F. Building usability knowledge for health information technology: a usability-oriented analysis of incident reports. Appl Clin Inform 2019; 10 (03) 395-408
• 23 Fong A, Komolafe T, Adams KT, Cohen A, Howe JL, Ratwani RM. Exploration and initial development of text classification models to identify health information technology usability-related patient safety event reports. Appl Clin Inform 2019; 10 (03) 521-527
• 24 Likourezos A, Chalfin DB, Murphy DG, Sommer B, Darcy K, Davidson SJ. Physician and nurse satisfaction with an electronic medical record system. J Emerg Med 2004; 27 (04) 419-424
• 25 Vehko T, Hyppönen H, Puttonen S. , et al. Experienced time pressure and stress: electronic health records usability and information technology competence play a role. BMC Med Inform Decis Mak 2019; 19 (01) 160
• 26 Kutney-Lee A, Sloane DM, Bowles KH, Burns LR, Aiken LH. Electronic health record adoption and nurse reports of usability and quality of care: the role of work environment. Appl Clin Inform 2019; 10 (01) 129-139
• 27 Topaz M, Ronquillo C, Peltonen LM. , et al. Nurse informaticians report low satisfaction and multi-level concerns with electronic health records: results from an international survey. AMIA Annu Symp Proc 2017; 2016: 2016-2025
• 28 Schottenfeld L, Petersen D, Peikes D. , et al. Creating Patient-Centered Team-Based Primary Care. AHRQ Pub. No. 16–0002-EF. Rockville, MD: Agency for Healthcare Research and Quality; 2016
• 29 Middleton B, Bloomrosen M, Dente MA. , et al; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1): e2-e8
• 30 Office of the National Coordinator for Health Information Technology (ONC), Department of Health and Human Services. Health information technology: standards, implementation specifications, and certification criteria for electronic health record technology, 2014 edition; revisions to the permanent certification program for health information technology. Final rule. Fed Regist 2012; 77 (171) 54163-54292
• 31 Ratwani RM, Reider J, Singh H. A decade of health information technology usability challenges and the path forward. JAMA 2019; 321 (08) 743-744
• 32 Shaw RJ, Horvath MM, Leonard D, Ferranti JM, Johnson CM. Developing a user-friendly interface for a self-service healthcare research portal: cost-effective usability testing. Health Syst (Basingstoke) 2015; 4 (02) 151-158
• 33 Belden J, Patel J, Lowrance N. , et al. Inspired EHRs: designing for clinicians. Columbia, MO: University of Missouri School of Medicine; 2014. Accessed May 17, 2020 at: http://inspiredEHRs.org
• 34 FDA. Applying human factors and usability engineering to medical devices: guidance for industry and food and drug administration staff. Office of Medical Products and Tobacco, Center for Devices and Radiological Health; 2016. Accessed July 30, 2020 at: https://www.fda.gov/media/80481/download
• 35 Lowry SZ, Quinn MT, Ramaiah M. , et al. Technical evaluation, testing, and validation of the usability of electronic health records. National Institute of Standards and Technology; 2012. Accessed July 30, 2020 at: https://www.nist.gov/publications/nistir-7804-technical-evaluation-testing-and-validation-usability-electronic-health
• 36 Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr 2013; 13 (6, Suppl): S38-S44