Download PDF
Appl Clin Inform 2022; 13(05): 1015-1023
DOI: 10.1055/a-1942-6889
Research Article

Effect of Notes' Access and Complexity on OpenNotes' Utility

Amro Khasawneh 
1   Industrial Engineering Department, School of Engineering, Mercer University, Macon, Georgia, United States
,
Ian Kratzke
2   Department of Surgery, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
,
Karthik Adapa
3   Department of Radiation Oncology, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
,
Lawrence Marks
3   Department of Radiation Oncology, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
,
Lukasz Mazur
3   Department of Radiation Oncology, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints

Abstract

Background Health care providers are now required to provide their patients access to their consultation and progress notes. Early research of this concept, known as “OpenNotes,” showed promising results in terms of provider acceptability and patient adoption, yet objective evaluations relating to patients' interactions with the notes are limited.

Objectives To assess the effect of the complexity level of notes and number of accesses (initial read vs. continuous access) on the user's performance, perceived usability, cognitive workload, and satisfaction with the notes.

Methods We used a 2*2 mixed subjects experimental design with two independent variables: (1) note's complexity at two levels (simple vs. complex) and (2) number of accesses to notes at two levels (initial vs. continuous). Fifty-three participants were randomly assigned to receive a simple versus complex radiation oncology clinical note and were tested on their performance for understanding the note content after an initial read, and then with continuous access to the note. Performance was quantified by comparing each participant's answers to the ones developed by the research team and assigning a score of 0 to 100 based on participants' understanding of the notes. Usability, cognitive workload, and satisfaction scores of the notes were quantified using validated tools.

Results Performance for understanding was significantly better in simple versus complex notes with continuous access (p = 0.001). Continuous access to the notes was also positively associated with satisfaction scores (p = 0.03). The overall perceived usability, cognitive workload, and satisfaction scores were considered low for both simple and complex notes.

Conclusion Simplifying notes can improve understanding of notes for patients/families. However, perceived usability, cognitive workload, and satisfaction with even the simplified notes were still low. To make notes more useful for patients and their families, there is a need for dramatic improvements to the overall usability and content of the notes.

Keywords

electronic health records - testing - evaluation - notes

Protection of Human and Animal Subjects

Participation was voluntary and does not pose undue risk. All human participants read and signed the informed consent form and all needed information was given to them when deciding whether to participate in the study. This was a remote unmoderated study and was fully implemented in Qualtrics.[42] The study protocol was reviewed and approved by the University of North Carolina at Chapel Hill Institutional Review Board (IRB) under reference ID: 338124.


Supplementary Material



Publication History

Received: 09 June 2022

Accepted: 11 September 2022

Accepted Manuscript online:
14 September 2022

Article published online:
26 October 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 HealthIT.gov. Office-based physician electronic health record adoption. Accessed May 18, 2021 at: https://dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php
    PubMedGoogle Scholar
  • 2 Roehrs A, da Costa CA, Righi RD, de Oliveira KSF. Personal health records: a systematic literature review. J Med Internet Res 2017; 19 (01) e13
    CrossrefPubMedGoogle Scholar
  • 3 Delbanco T, Walker J, Darer JD. et al. Open notes: doctors and patients signing on. Ann Intern Med 2010; 153 (02) 121-125
    CrossrefPubMedGoogle Scholar
  • 4 Kayastha N, Pollak KI, LeBlanc TW. Open notes: a qualitative study of oncology patients' experiences reading their cancer care notes. J Clin Oncol 2017; 35 (31, suppl): 33-33
    CrossrefPubMedGoogle Scholar
  • 5 Murugan A, Gooding H, Greenbaum J. et al. Lessons learned from OpenNotes learning mode and subsequent implementation across a pediatric health system. Appl Clin Inform 2022; 13 (01) 113-122
    Article in Thieme ConnectPubMedGoogle Scholar
  • 6 Salmi L, Dong ZJ, Yuh B, Walker J, DesRoches CM. Open notes in oncology: patient versus oncology clinician views. Cancer Cell 2020; 38 (06) 767-768
    CrossrefPubMedGoogle Scholar
  • 7 Shaverdian N, Wang X, Hegde JV. et al. The patient's perspective on breast radiotherapy: initial fears and expectations versus reality. Cancer 2018; 124 (08) 1673-1681
    CrossrefPubMedGoogle Scholar
  • 8 Shaverdian N, Chang EM, Chu F-I. et al. Impact of open access to physician notes on radiation oncology patients: results from an exploratory survey. Pract Radiat Oncol 2019; 9 (02) 102-107
    CrossrefPubMedGoogle Scholar
  • 9 Turer RW, DesRoches CM, Salmi L, Helmer T, Rosenbloom ST. Patient perceptions of receiving COVID-19 test results via an online patient portal: an open results survey. Appl Clin Inform 2021; 12 (04) 954-959
    Article in Thieme ConnectPubMedGoogle Scholar
  • 10 Sarabu C, Lee T, Hogan A, Pageler N. The value of OpenNotes for pediatric patients, their families and impact on the patient-physician relationship. Appl Clin Inform 2021; 12 (01) 76-81
    Article in Thieme ConnectPubMedGoogle Scholar
  • 11 Ponathil AP, Khasawneh A, Byrne K, Madathil KC. Factors affecting the choice of a dental care provider by older adults based on online consumer reviews. IISE Trans Healthc Syst Eng 2021; 11 (01) 51-69
    CrossrefPubMedGoogle Scholar
  • 12 Blease C, Salmi L, DesRoches CM. Open notes in cancer care: coming soon to patients. Lancet Oncol 2020; 21 (09) 1136-1138
    CrossrefPubMedGoogle Scholar
  • 13 NIH. Cancer patients say clinical notes access valuable - National Cancer Institute. Accessed May 18, 2021 at: https://www.cancer.gov/news-events/cancer-currents-blog/2020/open-clinical-notes-access-by-cancer-patients
    PubMedGoogle Scholar
  • 14 Cohen J. The effect size. In: Statistical Power Analysis for the Behavioral Sciences. Mahwah: NJLawrence Erlbaum Associates; 1988: 8-13
    PubMedGoogle Scholar
  • 15 Khasawneh A, Chalil Madathil K, Dixon E, Wisniewski P, Zinzow H, Roth R. An investigation on the portrayal of blue whale challenge on youtube and twitter. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 887-888
    CrossrefPubMedGoogle Scholar
  • 16 Research for Me - Home. Accessed May 10, 2022 at https://researchforme.unc.edu/index.php/en/
    PubMedGoogle Scholar
  • 17 Harwood TG, Garry T. An overview of content analysis. Marketing Rev 2003; 3 (04) 479-498
    CrossrefPubMedGoogle Scholar
  • 18 Richards KAR, Hemphill MA. A practical guide to collaborative qualitative data analysis. J Teach Phys Educ 2018; 37 (02) 225-231
    CrossrefPubMedGoogle Scholar
  • 19 Khasawneh A, Chalil Madathil K, Dixon E, Wiśniewski P, Zinzow H, Roth R. Examining the self-harm and suicide contagion effects of the blue whale challenge on YouTube and Twitter: qualitative study. JMIR Ment Health 2020; 7 (06) e15973
    CrossrefPubMedGoogle Scholar
  • 20 Khasawneh A, Madathil KC, Zinzow H. et al. An investigation of the portrayal of social media challenges on youtube and twitter. Trans Soc Comput 2021; 4 (01) 1-23
    CrossrefPubMedGoogle Scholar
  • 21 Peres SC, Pham T, Phillips R. Validation of the system usability scale (SUS). Proc Hum Factors Ergon Soc Annu Meet 2013; 57 (01) 192-196
    CrossrefPubMedGoogle Scholar
  • 22 Wilson MK, Khasawneh A, Ponathil A. et al. A preliminary study investigating patients' perceptions of research consenting methods. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 1931-1935
    CrossrefPubMedGoogle Scholar
  • 23 Khasawneh A, Rogers H, Bertrand J, Madathil KC, Gramopadhye A. Human adaptation to latency in teleoperated multi-robot human-agent search and rescue teams. Autom Construct 2019; 99: 265-277
    CrossrefPubMedGoogle Scholar
  • 24 Gomes KM, Ratwani RM. Evaluating improvements and shortcomings in clinician satisfaction with electronic health record usability. JAMA Netw Open 2019; 2 (12) e1916651
    CrossrefPubMedGoogle Scholar
  • 25 Cole AC, Adapa K, Khasawneh A, Richardson DR, Mazur L. Codesign approaches involving older adults in the development of electronic healthcare tools: a systematic review. BMJ Open 2022; 12 (07) e058390
    CrossrefPubMedGoogle Scholar
  • 26 Lewis JR, Sauro J. Item benchmarks for the system usability scale. Journal of Usability Studies 2018; 13 (03) 158-167
    PubMedGoogle Scholar
  • 27 Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research. Amsterdam: Elsevier; 2004
    Google Scholar
  • 28 Hart SG. Nasa-Task Load Index (NASA-TLX); 20 years later. Proc Hum Factors Ergon Soc Annu Meet 2006; 50 (09) 904-908
    CrossrefPubMedGoogle Scholar
  • 29 Hamad J, Fox A, Kammire MS, Hollis AN, Khairat S. Evaluating the experiences of new and existing teledermatology patients during the COVID-19 pandemic: cross-sectional survey study. JMIR Dermatol 2021; 4 (01) e25999
    CrossrefPubMedGoogle Scholar
  • 30 Cowan N. What are the differences between long-term, short-term, and working memory?. Prog Brain Res 2008; 169: 323-338
    Google Scholar
  • 31 Mishra VK, Hoyt RE, Wolver SE, Yoshihashi A, Banas C. Qualitative and quantitative analysis of patients' perceptions of the patient portal experience with OpenNotes. Appl Clin Inform 2019; 10 (01) 10-18
    Article in Thieme ConnectPubMedGoogle Scholar
  • 32 Fossa AJ, Bell SK, DesRoches C. OpenNotes and shared decision making: a growing practice in clinical transparency and how it can support patient-centered care. J Am Med Inform Assoc 2018; 25 (09) 1153-1159
    CrossrefPubMedGoogle Scholar
  • 33 Leveille SG, Walker J, Ralston JD, Ross SE, Elmore JG, Delbanco T. Evaluating the impact of patients' online access to doctors' visit notes: designing and executing the OpenNotes project. BMC Med Inform Decis Mak 2012; 12: 32
    CrossrefPubMedGoogle Scholar
  • 34 Bialostozky M, Huang JS, Kuelbs CL. Are you in or are you out? provider note sharing in pediatrics. Appl Clin Inform 2020; 11 (01) 166-171
    Article in Thieme ConnectPubMedGoogle Scholar
  • 35 Adobe XD Ideas. The System Usability Scale & How it's Used in UX. Accessed February 7, 2022 at: https://xd.adobe.com/ideas/process/user-testing/sus-system-usability-scale-ux/
    PubMedGoogle Scholar
  • 36 Usability.gov. System Usability Scale (SUS). Accessed February 8, 2022 at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
    PubMedGoogle Scholar
  • 37 Prabaswari AD, Basumerda C, Utomo BW. The mental workload analysis of staff in study program of private educational organization. IOP Conf Ser: Mater Sci Eng 2019; 528: 012018
    CrossrefPubMedGoogle Scholar
  • 38 Walker J, Leveille S, Bell S. et al. Opennotes after 7 years: patient experiences with ongoing access to their clinicians' outpatient visit notes. J Med Internet Res 2019; 21 (05) e13876
    CrossrefPubMedGoogle Scholar
  • 39 Panattoni L, Stone A, Chung S, Tai-Seale M. Patients report better satisfaction with part-time primary care physicians, despite less continuity of care and access. J Gen Intern Med 2015; 30 (03) 327-333
    CrossrefPubMedGoogle Scholar
  • 40 Alpert JM, Morris BB, Thomson MD, Matin K, Geyer CE, Brown RF. OpenNotes in oncology: oncologists' perceptions and a baseline of the content and style of their clinician notes. Transl Behav Med 2019; 9 (02) 347-356
    CrossrefPubMedGoogle Scholar
  • 41 Joseph A, Chalil Madathil K, Jafarifiroozabadi R. et al. Communication and teamwork during telemedicine-enabled stroke care in an ambulance. Hum Factors 2022; 64 (01) 21-41
    CrossrefPubMedGoogle Scholar
  • 42 QualtricsXM . The leading experience management software. Accessed November 2, 2021 at: https://www.qualtrics.com/
    PubMedGoogle Scholar