Usability Assessment of Secure Messaging for Clinical Document Sharing between Health Care Providers and Patients

Abstract

Background Web-based patient portals feature secure messaging systems that enable health care providers and patients to communicate information. However, little is known about the usability of these systems for clinical document sharing.

Objective This article evaluates the usability of a secure messaging system for providers and patients in terms of its ability to support sharing of electronic clinical documents.

Methods We conducted usability testing with providers and patients in a human–computer interaction laboratory at a Midwestern U.S. hospital. Providers sent a medication list document to a fictitious patient via secure messaging. Separately, patients retrieved the clinical document from a secure message and returned it to a fictitious provider. We collected use errors, task completion, task time, and satisfaction.

Results Twenty-nine individuals participated: 19 providers (6 physicians, 6 registered nurses, and 7 pharmacists) and 10 patients. Among providers, 11 (58%) attached and sent the clinical document via secure messaging without requiring assistance, in a median (range) of 4.5 (1.8–12.7) minutes. No patients completed tasks without moderator assistance. Patients accessed the secure messaging system within 3.6 (1.2–15.0) minutes; retrieved the clinical document within 0.8 (0.5–5.7) minutes; and sent the attached clinical document in 6.3 (1.5–18.1) minutes. Although median satisfaction ratings were high, with 5.8 for providers and 6.0 for patients (scale, 0–7), we identified 36 different use errors. Physicians and pharmacists requested additional features to support care coordination via health information technology, while nurses requested features to support efficiency for their tasks.

Conclusion This study examined the usability of clinical document sharing, a key feature of many secure messaging systems. Our results highlight similarities and differences between provider and patient end-user groups, which can inform secure messaging design to improve learnability and efficiency. The observations suggest recommendations for improving the technical aspects of secure messaging for clinical document sharing.

Keywords

personal health records - secure messaging - communication - interfaces and usability - electronic health records and systems

Protection of Human and Animal Subjects

This study was performed in compliance with standards set by the Institutional Review Boards at Indiana University and the VA Research and Development Committee. These organizations reviewed and approved the research methods.

Authors' Contributions

A.R. led the grant writing and study design, with contributions from A.Z., A.L., and S.S. M.J. developed the tasks scenarios with input from coauthors; moderated most usability sessions; conducted the analyses for use errors; and drafted the initial manuscript. B.P. recruited patient participants; assisted with demographic, task completion, and efficiency data analyses; and drafted most figures and tables. H.P. moderated some usability sessions and analyzed the satisfaction data. All authors contributed to and approved the final version of the manuscript.

• References

• 1 Cronin RM, Davis SE, Shenson JA, Chen Q, Rosenbloom ST, Jackson GP. Growth of secure messaging through a patient portal as a form of outpatient interaction across clinical specialties. Appl Clin Inform 2015; 6 (02) 288-304
• 2 Heyworth L, Clark J, Marcello TB. , et al. Aligning medication reconciliation and secure messaging: qualitative study of primary care providers' perspectives. J Med Internet Res 2013; 15 (12) e264
• 3 Shenson JA, Cronin RM, Davis SE, Chen Q, Jackson GP. Rapid growth in surgeons' use of secure messaging in a patient portal. Surg Endosc 2016; 30 (04) 1432-1440
• 4 Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res 2015; 17 (02) e40
• 5 Haun JN, Patel NR, Lind JD, Antinori N. Large-scale survey findings inform patients' experiences in using secure messaging to engage in patient-provider communication and self-care management: a quantitative assessment. J Med Internet Res 2015; 17 (12) e282
• 6 Henry J, Pylypchuk Y, Patel V. Electronic capabilities for patient engagement among US non-federal acute care hospitals: 2012–2015. The Office of the National Coordinator for Health Information Technology; 2016
• 7 Schnipper JL, Gandhi TK, Wald JS. , et al. Design and implementation of a web-based patient portal linked to an electronic health record designed to improve medication safety: the Patient Gateway medications module. Inform Prim Care 2008; 16 (02) 147-155
• 8 Heyworth L, Paquin AM, Clark J. , et al. Engaging patients in medication reconciliation via a patient portal following hospital discharge. J Am Med Inform Assoc 2014; 21 (e1): e157-e162
• 9 Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc 2006; 13 (02) 121-126
• 10 Chow CK, Redfern J, Hillis GS. , et al. Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: a randomized clinical trial. JAMA 2015; 314 (12) 1255-1263
• 11 Osborn CY, Mayberry LS, Wallston KA, Johnson KB, Elasy TA. Understanding patient portal use: implications for medication management. J Med Internet Res 2013; 15 (07) e133
• 12 Kuo A, Dang S. Secure messaging in electronic health records and its impact on diabetes clinical outcomes: a systematic review. Telemed J E Health 2016; 22 (09) 769-777
• 13 Green BB, Cook AJ, Ralston JD. , et al. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. JAMA 2008; 299 (24) 2857-2867
• 14 Raghu TS, Frey K, Chang Y-H, Cheng M-R, Freimund S, Patel A. Using secure messaging to update medications list in ambulatory care setting. Int J Med Inform 2015; 84 (10) 754-762
• 15 Haun JN, Lind JD, Shimada SL. , et al. Evaluating user experiences of the secure messaging tool on the Veterans Affairs' patient portal system. J Med Internet Res 2014; 16 (03) e75
• 16 Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res 2015; 17 (06) e148
• 17 Ferranti JM, Musser RC, Kawamoto K, Hammond WE. The clinical document architecture and the continuity of care record: a critical analysis. J Am Med Inform Assoc 2006; 13 (03) 245-252
• 18 Choi M, Lee HS, Park JH. Usability of academic electronic medical record application for nursing students' clinical practicum. Healthc Inform Res 2015; 21 (03) 191-195
• 19 Russ AL, Jahn MA, Patel H. , et al. Usability evaluation of a medication reconciliation tool: embedding safety probes to assess users' detection of medication discrepancies. J Biomed Inform 2018;
• 20 Russ AL, Weiner M, Russell SA, Baker DA, Fahner WJ, Saleem JJ. Design and implementation of a hospital-based usability laboratory: insights from a Department of Veterans Affairs laboratory for health information technology. Jt Comm J Qual Patient Saf 2012; 38 (12) 531-540
• 21 Nielsen J. Demonstrate Thinking Aloud by Showing Users a Video. Available at: https://www.nngroup.com/articles/thinking-aloud-demo-video2014 . Accessed February 23, 2017
• 22 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
• 23 Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC. Six-item screener to identify cognitive impairment among potential subjects for clinical research. Med Care 2002; 40 (09) 771-781
• 24 Nielsen J. Why you only need to test with 5 users. Useit Com Alertbox; 2000
• 25 FDA. Applying Human Factors and Usability Engineering to Medical Devices. Silver Spring, MA: Food and Drug Administration; 2016
• 26 Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995; 7 (01) 57-78
• 27 Cooper A, Reimann R, Cronin D. About face 3: the essentials of interaction design. John Wiley & Sons; 2007
• 28 The VA and Academic Medicine: Partners in Health Care, Training, and Research. (2014). Available at: https://www.aamc.org/download/385612/data/07182014.pdf
• 29 Thompson MJ, Reilly JD, Valdez RS. Work system barriers to patient, provider, and caregiver use of personal health records: A systematic review. Appl Ergon 2016; 54: 218-242
• 30 Urowitz S, Wiljer D, Dupak K. , et al. Improving diabetes management with a patient portal: a qualitative study of diabetes self-management portal. J Med Internet Res 2012; 14 (06) e158
• 31 Jahn M, Caldwell BS. Developing Distributed Expertise Coordination Models of Diabetes Care Teams. In: Proceedings of 2018 Human Factors and Ergonomics Society Annual Meeting, Philadelphia, PA (In Press)
• 32 U.S. Department of Veterans Affairs. VA Utilization Profile. Web 2017; 2017: 20