Background and Significance
Background and Significance
Patient portals allow patients to track test results, report self-administered medications, and communicate with their providers through portal-mediated secure messages (PSMs).[1] Oncology patients rate this opportunity to engage with their care team as more important than other patients, likely related to the increased complexity of their management.[2] PSMs provide a valuable source of information about patients, including references to health needs, care coordination, and questions about their treatment plans.[3]
[4]
Some patients prefer that their caregivers have access to their patient portal to help coordinate care and overcome barriers such as geographic distance.[5] This has led to the development of registered proxy accounts or the setup of separate, distinct accounts for caregivers.[6]
[7] However, an estimated less than 1% of caregivers utilize formal proxy accounts because it is more convenient for them to use the patient's account.[5] As such, PSMs are frequently not authored by the patient, requiring careful interpretation of the messages by the care team.[5]
[8] A study found that 46% of adult diabetic patients had at least one PSM authored by a caregiver who used the patient's portal credentials to log in (unregistered proxy) instead of using their proxy account.[9] Identification of unregistered proxy users can enhance patient security through accurate authentication, ensure that needs expressed in PSMs are appropriately being responded to whether from patients or caregivers, and help systems leverage technology to address barriers to portal engagement.
Little is known about the prevalence of unregistered proxy use among oncology patients or the demographics of oncology patients that may have others send PSMs on their behalf. This study aimed to determine if sociodemographic differences exist between patients with a high number of presumed-unregistered proxy messages compared with those with few to no unregistered messages in our patient population.
Methods
This study took place at Memorial Sloan Kettering Cancer Center (MSK), a National Comprehensive Cancer Network center with seven regional campuses in New York. MSK's homegrown patient portal system, MyMSK, started enrollment in 2006 and began supporting registered proxy accounts in 2017. MSK provides an optimal environment for investigation given the high rate of patient engagement. As of 2022, 89% of active patients, those who received treatment within the past year, had MyMSK accounts with 13% of patients with a MyMSK account having at least one formally registered proxy account. To be included in our study, PSMs must have been sent from a patient or registered proxy account to a physician's inbox after December 31, 2016. Additionally, the PSM had to be the first message in a conversation chain. A total of 1.5 million PSMs met these criteria. Our cohort was randomly selected based on the power of our available computational resources.
The use of natural language processing (NLP) models to extract data from unstructured free text in the electronic health record (EHR) has demonstrated efficiency and potential to reduce missed clinically significant information compared with manual review.[10]
[11] The complete methodology is described in a manuscript detailing the development of our NLP model to classify the authorship of PSMs.[12] In brief, we trained an NLP model on a dataset of 1,850 individual messages with manual annotation of patient or presumed proxy as our gold standard in concordance with other NLP models categorizing PSMs.[3]
[13]
[14] This model achieved an AUC of 0.93. To then categorize a patient's account, we determined how often a true proxy user's messages predict as proxy-written utilizing the model. Running the model on a new set of messages (training data), all written by registered proxy users determined that, on average, a confirmed registered proxy only had 26.1% of messages that were flagged by the model. We then performed further analyses to select a cut-off value of 0.25, which maximized accuracy and sensitivity of the model.[12] Given that unregistered proxy accounts likely behave linguistically similar to registered proxy accounts, the threshold for classifying a user as an unregistered proxy was set at a positive prediction for 25% of their messages. We then applied the model to messages from 12,119 randomly selected patients.
Our primary statistical analyses involve comparing differences in self-reported sociodemographic variables between accounts that were categorized by the model as presumed proxy and those that were not. Continuous variables are summarized using median and interquartile range (IQR, 25th and 75th percentiles) and compared using the Wilcoxon rank sum test. Categorical variables are described using count and percentage and compared using the chi-square test.
Results
Our NLP model classified 12,119 individual messages from unique patients, of which 1,253 (10.3%) appeared to be sent by someone other than the patient. Patient demographics comparing predicted proxy and nonproxy users are summarized in [Table 1]. There was no difference in ethnicity or marital status between cohorts. Patients predicted to have proxies sending PSM were significantly more likely to be male, older (68 vs. 61 years old), and identify as a nonwhite race. Additionally, there was a higher representation of patients with a non-English preferred language, noncitizens, and public insurance type among presumed proxy accounts.
Table 1
Sociodemographics differences for patients with no proxy messages compared with those with a presumed proxy
|
Predicted proxy
|
|
|
0 (N = 10,863)
|
1 (N = 1,256)
|
Total (N = 12,119)
|
p-Value
|
|
Patient sex
|
|
Female
|
06299 (57.99%)
|
00591 (47.05%)
|
06890 (56.85%)
|
<0.001[a]
|
|
Male
|
04564 (42.01%)
|
00665 (52.95%)
|
05229 (43.15%)
|
|
Race
|
|
Asian
|
00724 (06.97%)
|
00099 (08.41%)
|
00823 (07.12%)
|
0.010[a]
|
|
Black
|
00517 (04.98%)
|
00065 (05.52%)
|
00582 (05.03%)
|
|
Other
|
00288 (02.77%)
|
00048 (04.08%)
|
00336 (02.90%)
|
|
White
|
08861 (85.28%)
|
00965 (81.99%)
|
09826 (84.95%)
|
|
Unknown
|
00473 (04.35%)
|
00079 (06.29%)
|
00552 (04.55%)
|
|
Ethnicity
|
|
Hispanic
|
00595 (05.74%)
|
00078 (06.62%)
|
00673 (05.83%)
|
0.222
|
|
Non-Hispanic
|
09778 (94.26%)
|
01101 (93.38%)
|
10879 (94.17%)
|
|
Unknown
|
00490 (04.51%)
|
00077 (06.13%)
|
00567 (04.68%)
|
|
Marital status
|
|
Married/life partner
|
07456 (70.83%)
|
00849 (71.05%)
|
08305 (70.85%)
|
0.875
|
|
Not married
|
03071 (29.17%)
|
00346 (28.95%)
|
03417 (29.15%)
|
|
Unknown
|
00336 (03.09%)
|
00061 (04.86%)
|
00397 (03.28%)
|
|
Preferred language
|
|
English
|
10450 (97.44%)
|
01133 (91.82%)
|
11583 (96.86%)
|
<0.001[a]
|
|
Non-English
|
00275 (02.56%)
|
00101 (08.18%)
|
00376 (03.14%)
|
|
Unknown
|
00138 (01.27%)
|
00022 (01.75%)
|
00160 (01.32%)
|
|
Age
|
|
Median (IQR)
|
61.0 (51.0–70.0)
|
68.0 (56.0–78.0)
|
62.0 (51.0–71.0)
|
<0.001[a]
|
|
Range
|
15.00–98.00
|
5.00–99.00
|
5.00–99.00
|
|
n
|
10863
|
1256
|
12119
|
|
Insurance type
|
|
Other
|
00839 (07.77%)
|
00135 (10.80%)
|
00974 (08.08%)
|
<0.001[a]
|
|
Private
|
05400 (49.99%)
|
00373 (29.84%)
|
05773 (47.90%)
|
|
Public
|
04563 (42.24%)
|
00742 (59.36%)
|
05305 (44.02%)
|
|
Unknown
|
00061 (00.56%)
|
00006 (00.48%)
|
00067 (00.55%)
|
Abbreviation: IQR, interquartile range.
a Statistically significant.
Discussion
Our results showed that oncology patients with a presumed unregistered proxy were more likely to be older, nonwhite, and have a non-English preferred language. This is in concordance with known disparities in portal use and associated PSM studies in a diabetic patient cohort where racial minorities, older patients, and patients with limited English proficiency were more likely to have unregistered proxy-written PSM.[7] In contrast, an analysis from Mayo Clinic found that up to 16% of PSMs were estimated to be sent by an unregistered proxy, and no significant differences in race or age were noted.[6] This may reflect the lack of racial and ethnic diversity in their study, with over 90% of patients identifying as white.
Patient EHR access is linked to improved patient satisfaction and outcomes through the promotion of patient engagement and shared decision-making.[15]
[16]
[17]
[18] PSMs allow for opportunities for further communication, engagement, and education, which has been reported to be particularly valuable among oncology patients.[19] Engaging caregivers offers a clinical benefit to patients, with joint access to patient portals enhancing discussion and agreement about care plans.[20]
[21] However, there are multiple requirements for patients to benefit from patient portal use. Patients/and or their proxies need internet access, computer literacy, and health literacy.[15]
[21] Recognizing the impact of demographics on the utilization of portals and messaging is important to improve the inclusiveness of the patient portal design.
Barriers to the uptake of patient portal use extend to the creation of proxy accounts with inequitable utilization worsening privacy and security for vulnerable groups if not addressed. Some portals offer variations in the degree of information access via proxy accounts, but controlling these various settings may involve increasing complexity for patients and their families.[22]
[23] Proxy account setup should include easy-to-follow instructions and technical assistance so that all patients can equitably access important security features. Account setup should also include options to give proxies full access to information, if the patient consents, to mitigate the need for proxies logging into the patient account to view needed information, which can result in confusion for the medical team and issues related to detecting true account security breaches. Preventing breaches in patient confidentiality is key to maintaining trust in medical practice and alleviating concerns regarding the privacy of health information.[16]
[24]
Also, needs expressed in the PSMs may reflect the needs of a caregiver rather than the patient. The burden experienced by cancer caregivers disproportionately impacts black and Hispanic caregivers with more caregiving tasks performed, less social support, and greater financial burden.[25]
[26] PSM (from registered and unregistered proxy accounts) may provide an existing resource for identifying specific challenges faced by caregivers that can inform intervention development to support those most in need. Further, reliably distinguishing between message authors is critical to allow for appropriate response or action and for ongoing efforts to utilize machine learning to extract information from PSM, such as patient-reported outcomes.[27]
This study reviewed PSMs sent by patients receiving treatment at a specialized cancer center with an established patient portal. As such, these results may differ from other medical settings with less complex care and may not generalize to all populations. Additionally, reliance on pronoun use to manually annotate messages may have inadvertently misclassified non-English speakers or those with lower education levels. Messages sent by a proxy using first-person singular pronouns (i.e., being intentionally deceptive and posing as the patient) would not have been identified. Utilization of race data is limited by accuracy and completeness in the EHR.[28] Further investigation is necessary to assess interventions to decrease unregistered proxies and ensure that patient portal platforms provide the support necessary for all patients to engage meaningfully and securely. Future work should incorporate clinical characteristics, caregiver sociodemographics, examine differences between registered and unregistered proxies, and consider the impact on patient outcomes.
Conclusion
Sociodemographic differences in patients with unregistered proxy users highlight groups that may not be well supported by the current functionality of patient portals. Identification of unregistered proxies provides an opportunity to develop a process to educate caregivers about registered accounts and determine if new initiatives are necessary to assist caregivers. For diverse patients to benefit from patient portals, portal optimization and further assessment of barriers are required.
