A Natural Language Processing Model to Identify Confidential Content in Adolescent Clinical Notes

Abstract

Background The 21st Century Cures Act mandates the immediate, electronic release of health information to patients. However, in the case of adolescents, special consideration is required to ensure that confidentiality is maintained. The detection of confidential content in clinical notes may support operational efforts to preserve adolescent confidentiality while implementing information sharing.

Objectives This study aimed to determine if a natural language processing (NLP) algorithm can identify confidential content in adolescent clinical progress notes.

Methods A total of 1,200 outpatient adolescent progress notes written between 2016 and 2019 were manually annotated to identify confidential content. Labeled sentences from this corpus were featurized and used to train a two-part logistic regression model, which provides both sentence-level and note-level probability estimates that a given text contains confidential content. This model was prospectively validated on a set of 240 progress notes written in May 2022. It was subsequently deployed in a pilot intervention to augment an ongoing operational effort to identify confidential content in progress notes. Note-level probability estimates were used to triage notes for review and sentence-level probability estimates were used to highlight high-risk portions of those notes to aid the manual reviewer.

Results The prevalence of notes containing confidential content was 21% (255/1,200) and 22% (53/240) in the train/test and validation cohorts, respectively. The ensemble logistic regression model achieved an area under the receiver operating characteristic of 90 and 88% in the test and validation cohorts, respectively. Its use in a pilot intervention identified outlier documentation practices and demonstrated efficiency gains over completely manual note review.

Conclusion An NLP algorithm can identify confidential content in progress notes with high accuracy. Its human-in-the-loop deployment in clinical operations augmented an ongoing operational effort to identify confidential content in adolescent progress notes. These findings suggest NLP may be used to support efforts to preserve adolescent confidentiality in the wake of the information blocking mandate.

Keywords

confidentiality - patient portals - natural language processing - machine learning - health information exchange

Protection of Human and Animal Subjects

The presented work was performed as part of a quality improvement effort at our institution and does not qualify as human subjects research.

Publication History

Received: 12 October 2022

Accepted: 01 March 2023

Accepted Manuscript online:
10 March 2023

Article published online:
24 May 2023

• References

• 1 Office of the National Coordinator for Health Information Technology. 21st Century Cures Act: interoperability, information blocking, and the ONC health IT certification program [Internet]. 2020. Accessed July 22, 2022 at: https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification
• 2 Holmgren AJ, Patel V, Charles D, Adler-Milstein J. US hospital engagement in core domains of interoperability. Am J Manag Care 2016; 22 (12) e395-e402
• 3 Nazi KM, Turvey CL, Klein DM, Hogan TP, Woods SSVA. VA OpenNotes: exploring the experiences of early patient adopters with access to clinical notes. J Am Med Inform Assoc 2015; 22 (02) 380-389
• 4 Mishra VK, Hoyt RE, Wolver SE, Yoshihashi A, Banas C. Qualitative and quantitative analysis of patients' perceptions of the patient portal experience with OpenNotes. Appl Clin Inform 2019; 10 (01) 10-18
• 5 Delbanco T, Walker J, Bell SK. et al. Inviting patients to read their doctors' notes: a quasi-experimental study and a look ahead. Ann Intern Med 2012; 157 (07) 461-470
• 6 Walker J, Leveille S, Bell S. et al. OpenNotes after 7 years: patient experiences with ongoing access to their clinicians' outpatient visit notes. J Med Internet Res 2019; 21 (05) e13876
• 7 Wright E, Darer J, Tang X. et al. Sharing physician notes through an electronic portal is associated with improved medication adherence: quasi-experimental study. J Med Internet Res 2015; 17 (10) e226
• 8 Pageler NM, Webber EC, Lund DP. Implications of the 21st Century Cures Act in pediatrics. Pediatrics 2021; 147 (03) e2020034199
• 9 Carlson J, Goldstein R, Hoover K, Tyson N. NASPAG/SAHM Statement: the 21st Century Cures Act and adolescent confidentiality. J Adolesc Health 2021; 68 (02) 426-428
• 10 Schapiro NA, Mihaly LK. The 21st Century Cures Act and challenges to adolescent confidentiality. J Pediatr Health Care 2021; 35 (04) 439-442
• 11 Reddy DM, Fleming R, Swain C. Effect of mandatory parental notification on adolescent girls' use of sexual health care services. JAMA 2002; 288 (06) 710-714
• 12 Vukadinovich DM. Minors' rights to consent to treatment: navigating the complexity of State laws. J Health Law 2004; 37 (04) 667-691
• 13 Pathak PR, Chou A. Confidential care for adolescents in the U.S. health care system. J Patient Cent Res Rev 2019; 6 (01) 46-50
• 14 Pampati S, Liddon N, Dittus PJ, Adkins SH, Steiner RJ. Confidentiality matters but how do we improve implementation in adolescent sexual and reproductive health care?. J Adolesc Health 2019; 65 (03) 315-322
• 15 Sharko M, Jameson R, Ancker JS, Krams L, Webber EC, Rosenbloom ST. State-by-state variability in adolescent privacy laws. Pediatrics 2022; 149 (06) e2021053458
• 16 Ginsburg KR, Slap GB, Cnaan A, Forke CM, Balsley CM, Rouselle DM. Adolescents' perceptions of factors affecting their decisions to seek health care. JAMA 1995; 273 (24) 1913-1918
• 17 Ford CA, Millstein SG, Halpern-Felsher BL, Irwin Jr CE. Influence of physician confidentiality assurances on adolescents' willingness to disclose information and seek future health care. A randomized controlled trial. JAMA 1997; 278 (12) 1029-1034
• 18 Lothen-Kline C, Howard DE, Hamburger EK, Worrell KD, Boekeloo BO. Truth and consequences: ethics, confidentiality, and disclosure in adolescent longitudinal prevention research. J Adolesc Health 2003; 33 (05) 385-394
• 19 Arvisais-Anhalt S, Lau M, Lehmann CU. et al. The 21st Century Cures Act and multiuser electronic health record access: potential pitfalls of information release. J Med Internet Res 2022; 24 (02) e34085
• 20 Parsons CR, Hron JD, Bourgeois FC. Preserving privacy for pediatric patients and families: use of confidential note types in pediatric ambulatory care. J Am Med Inform Assoc 2020; 27 (11) 1705-1710
• 21 Bedgood M, Kuelbs CL, Jones VG, Pageler N. Organizational perspectives on technical capabilities and barriers related to pediatric data sharing and confidentiality. JAMA Netw Open 2022; 5 (07) e2219692
• 22 Bedgood M, Rabbani N, Brown C. et al. The prevalence of confidential content in adolescent progress notes prior to the 21st Century Cures Act information blocking mandate. Appl Clin Inform 2023
• 23 Ramos J. Using TF-IDF to determine word relevance in document queries. In: Proceedings of the First Instructional Conference on Machine Learning. International Conference on Machine Learning; 2003
• 24 Ng AY. Feature selection, L1 vs. L2 regularization, and rotational invariance. In: Proceedings of the Twenty-First International Conference on Machine Learning; 2004 78.
• 25 Ip W, Yang S, Parker J. et al. Assessment of Prevalence of Adolescent Patient Portal Account Access by Guardians. JAMA Netw Open 2021; 4 (09) e2124733
• 26 Xie J, McPherson T, Powell A. et al. Ensuring adolescent patient portal confidentiality in the Age of the Cures Act final rule. J Adolesc Health 2021; 69 (06) 933-939
• 27 Lee J, Yang S, Holland-Hall C. et al. Prevalence of sensitive terms in clinical notes using natural language processing techniques: observational study. JMIR Med Inform 2022; 10 (06) e38482
• 28 Ni Y, Bachtel A, Nause K, Beal S. Automated detection of substance use information from electronic health records for a pediatric population. J Am Med Inform Assoc 2021; 28 (10) 2116-2127
• 29 Murugan A, Gooding H, Greenbaum J. et al. Lessons learned from OpenNotes learning mode and subsequent implementation across a pediatric health system. Appl Clin Inform 2022; 13 (01) 113-122
• 30 Office of the National Coordinator for Health Information Technology. Cures Act final rule: information blocking exceptions [Internet]. 2022. Accessed September 12, 2022 at: https://www.healthit.gov/sites/default/files/2022-07/InformationBlockingExceptions.pdf
• 31 Campbell S, Giadresco K. Computer-assisted clinical coding: a narrative review of the literature on its benefits, limitations, implementation and impact on clinical coding professionals. HIM J 2020; 49 (01) 5-18
• 32 Woo M. An AI boost for clinical trials. Nature 2019; 573 (7775): S100-S102
• 33 Maguire H, Elana M, Roy R, Vivian L, Washington V, Volpp KG. Asked and answered: building a chatbot to address Covid-19-related concerns. Catalyst non issue content 2020. Accessed April 14, 2023 at: https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0230/
• 34 3M CodeAssist System [Internet]. Accessed September 12, 2022 at: https://www.3m.com/3M/en_US/health-information-systems-us/improve-revenue-cycle/coding/professional/code-assist/
• 35 Health Language. NLP for unstructured data [Internet]. Accessed September 12, 2022 at: https://www.wolterskluwer.com/en/solutions/health-language/clinical-natural-language-processing
• 36 Scibilia JP. How to protect maternal health information in newborn's medical record. AAP News 2014; 35 (12) 4-4
• 37 Spector-Bagdady K, Mello MM. Protecting the privacy of reproductive health information after the fall of Roe v Wade. JAMA Health Forum 2022; 3 (06) e222656-e222656