Usability and Utility of Human Immunodeficiency Virus Pre-exposure Prophylaxis Clinical Decision Support to Increase Knowledge and Pre-exposure Prophylaxis Initiations among Pediatric Providers

Keywords

clinical decision support - electronic health records - adolescent medicine - alert - order set - healthcare providers - pediatric health

Background and Significance

In 2020, one in five human immunodeficiency virus (HIV) diagnoses in the United States occurred in adolescents and young adults under the age of 25 years, disproportionately affecting gender-, sexual-, and racial/ethnic-minority youth.[1] Oral HIV preexposure prophylaxis (PrEP) is highly effective for preventing sexual transmission of HIV, reducing the risk of infection by 99%.[2] At this time, adolescents and young adults represent the group with the greatest unmet PrEP need compared with all other age groups. In 2019, only 15% of youth aged 16 to 24 years with indications for HIV prophylaxis were being prescribed PrEP.[3] This statistic has likely worsened as a result of the coronavirus disease 2019 pandemic.[4] Prior research demonstrated a 28% reduction in new PrEP users among adolescents and young adults during the first year of the pandemic; this was the largest decrease in PrEP initiations among all age groups.[4]

Lack of provider education and training has been repeatedly identified as one of the primary barriers to prescribing PrEP.[5] [6] [7] [8] [9] [10] [11] Previous research found that only one-third of primary care providers received any HIV-related training,[5] and most were unfamiliar with PrEP guidelines and unable to identify candidates for PrEP.[9] [10] [11] The lack of provider knowledge and experience is even greater when applied to adolescent patients compared to adult patients.[10] [12] However, prior studies have also demonstrated that providing even minimal on-the-job training and basic PrEP education can overcome knowledge and self-efficacy barriers.[9] [13] [14] [15] [16] [17] [18] [19] [20] Given the high rates of HIV and low rates of PrEP prescriptions among the youth population,[1] [7] [10] [21] there is an urgent need for more interventions to address provider barriers to PrEP prescribing.[5] [6] [7]

Electronic health record (EHR)-based clinical decision support systems (CDSSs) have the potential to increase evidence-based practice by integrating clinical guidelines and relevant patient data at the point of care to address possible provider gaps in training. However, the effectiveness of a CDSS largely depends on the usability and acceptability of the tool to fit within provider EHR workflows.[22] [23] [24] [25] Research evaluating the effect of CDSS on provider practices has had mixed findings.[26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] Studies on CDSS tools have shown interventions both successfully changing provider behavior[26] [27] [30] [31] [32] [35] [36] [37] [38] [39] and having no impact on provider practice.[28] [34] Previous research reported high levels of interest in PrEP-specific CDSS support among PrEP-inexperienced providers and found that a CDSS would have the greatest impact on knowledge and practice among providers without PrEP experience.[40] This is the first study of its kind to evaluate the utility of a PrEP CDSS to increase PrEP knowledge and likelihood to initiate PrEP among pediatric and adolescent providers.

Objectives

The purpose of this study was to evaluate changes in provider knowledge of, likelihood to prescribe, and the likelihood to refer a patient for PrEP after exposure to the PrEP CDSS. We hypothesized that providers would report improved PrEP knowledge and an increased likelihood to initiate PrEP after interacting with the CDSS.

A secondary objective was to explore the perceived provider utility of the CDSS and provider suggestions for improving PrEP CDSS effectiveness.

Methods

Study Design and Participants

This study was conducted within a single pediatric health care system that comprises a freestanding children's hospital and over 60 clinic service locations for pediatric primary and specialty care. In December 2019, a PrEP CDSS went live, which has been previously described.[41] The CDSS consisted of a hard stop alert question when an HIV test was ordered for patients 13 years and older that read, “Would this patient benefit from PrEP (a safe, daily pill to reduce HIV risk by ∼99%)?” ([Supplementary Appendix 1], available in the online version). If providers selected “yes'' or “not sure,” the CDSS would launch ([Supplementary Appendix 2], available in the online version) with options to (1) open a PrEP order set with labs, medication, patient education, and follow-up recommendations ([Supplementary Appendix 3], available in the online version), (2) refer their patient to an internal PrEP provider ([Supplementary Appendix 4], available in the online version), and/or (3) send a 15-minute educational module to their EHR message inbox ([Supplementary Appendix 5], available in the online version). The CDSS alert was removed from departments where HIV testing was being done for routine workups as opposed to targeted sexual behavior such as transplant, obstetrics, and reproductive endocrinology.

This was a prospective study using survey responses from a convenience sample of pediatric providers who launched the PrEP CDSS between June 1, 2020 and March 31, 2022.

All pediatric providers who launched the PrEP CDSS during the study period were sent a 17-question electronic survey within 7 days of launching the CDSS, which evaluated their experience with PrEP and the PrEP CDSS. Providers who completed the survey had an option to receive a $20 electronic gift card.

Survey Measures

The survey evaluated provider experiences with PrEP and the PrEP CDSS across multiple domains as described in [Tables 1] [2] [3] [4] to [5]. The survey also collected demographic information including profession (e.g., physician, nurse practitioner and physician assistant), area of practice (e.g., primary care and specialty care), years in practice, age, gender, and race/ethnicity. The intervention evaluation portion of the survey utilized questions adapted from the Centers for Disease Control and Prevention's “Recommended Training Effectiveness Questions for Postcourse Evaluations.”[42]

Table 1 Descriptive statistics of study population baseline characteristics

Participant characteristics	Total (%) (n = 43)
Age, mean years (SD)	40.4 (10.3)
Gender, n (%)
Female	28 (65.1)
Male	12 (27.9)
Prefer not to say	3 (7.0)
Race/ethnicity, n (%)
Asian/Pacific Islander	20 (46.5)
White	15 (34.9)
Another race/ethnicity	5 (11.6)
Prefer not to say	2 (4.7)
Hispanic/Latinx	1 (2.3)
Profession, n (%)
Physician	38 (90.5)
Nurse practitioner	3 (7.1)
Prefer not to say	1 (2.4)
Area of practice, n (%)
Primary care	27 (62.8)
Specialty care[a]	9 (20.9)
Adolescent medicine	6 (14.0)
Prefer not to say	1 (2.3)
Years in practice, mean (SD)	9.7 (9.8)
PrEP prescribing frequency, before CDSS alert, n (%)
Never	37 (86.1)
< 1 time/y	3 (7.0)
1–5 times/y	2 (4.7)
Prefer not to answer	1 (2.3)

Abbreviations: CDSS, clinical decision support system; PrEP, preexposure prophylaxis; SD, standard deviation.

^a Does not include adolescent medicine

Table 2 Change in provider-reported PrEP knowledge, likelihood to prescribe PrEP, and likelihood to refer a patient for PrEP before and after clinical decision support system (CDSS) exposure

Outcome	Provider Response (n = 43)					p-Value[a]
	Not at all knowledgeable	Slightly knowledgeable	Moderately knowledgeable	Very knowledgeable	Extremely knowledgeable
PrEP knowledge						0.0001
Before CDSS exposure	11 (25.6)	20 (46.5)	10 (23.3)	2 (4.7)	0 (0.0)
After CDSS exposure	0 (0.0)	15 (34.9)	19 (44.2)	7 (16.3)	0 (0.0)
	Extremely unlikely	Somewhat unlikely	Neither likely nor unlikely	Somewhat likely	Extremely likely
Likelihood to prescribe PrEP						<0.0001
Before CDSS exposure	19 (44.2)	7 (16.3)	13 (30.2)	2 (4.7)	2 (4.7)
After CDSS exposure	3 (7.0)	7 (16.3)	11 (25.6)	17 (39.5)	5 (11.6)
Likelihood to refer a patient for PrEP						<0.0001
Before CDSS exposure	9 (20.9)	5 (11.6)	11 (25.6)	11 (25.6)	9 (20.9)
After CDSS exposure	0 (0.0)	0 (0.0)	6 (14.0)	12 (27.9)	25 (58.1)

Abbreviations: CDSS, clinical decision support system; PrEP, preexposure prophylaxis.

^a p-Value calculated using McNemar's test comparing the marginal frequencies of providers who reported being at least “moderately knowledgeable” (compared to “not at all” or “slightly knowledgeable”) and at least “somewhat likely” to prescribe or refer for PrEP (compared to “extremely unlikely,” “somewhat unlikely,” or “neither likely nor unlikely”), before and after exposure to the CDSS.

Table 3 Change in pediatric primary care provider-reported PrEP knowledge, likelihood to prescribe PrEP, and likelihood to refer a patient for PrEP before and after clinical decision support system (CDSS) exposure

Outcome	Provider Response (n = 27)					p-Value[a]
	Not at all knowledgeable	Slightly knowledgeable	Moderately knowledgeable	Very knowledgeable	Extremely knowledgeable
PrEP knowledge						0.0002
Before CDSS exposure	9 (33.3)	14 (51.9)	3 (11.1)	1 (3.7)	0 (0.0)
After CDSS exposure	0 (0.0)	10 (37.0)	13 (48.2)	4 (14.8)	0 (0.0)
	Extremely unlikely	Somewhat unlikely	Neither likely nor unlikely	Somewhat likely	Extremely likely
Likelihood to prescribe PrEP						0.0001
Before CDSS exposure	15 (55.6)	6 (22.2)	6 (22.2)	0 (0.0)	0 (0.0)
After CDSS exposure	1 (3.7)	5 (18.5)	7 (25.9)	12 (44.4)	2 (7.4)
Likelihood to refer a patient for PrEP						0.0001
Before CDSS exposure	8 (29.6)	4 (14.8)	8 (29.6)	5 (18.5)	2 (7.4)
After CDSS exposure	0 (0.0)	0 (0.0)	3 (11.1)	9 (33.3)	15 (55.6)

Abbreviations: CDSS, clinical decision support system; PrEP, preexposure prophylaxis.

^a p-Value calculated using McNemar's test comparing the marginal frequencies of providers who reported being at least “moderately knowledgeable” (compared to “not at all” or “slightly knowledgeable”) and at least “somewhat likely” to prescribe or refer for PrEP (compared to “extremely unlikely,” “somewhat unlikely,” or “neither likely nor unlikely”), before and after exposure to the CDSS.

Table 4 Descriptive statistics of participant PrEP clinical decision support system (CDSS) experiences

PrEP prescribing and CDSS experience	Total (%) (n = 43)
PrEP CDSS was an effective educational tool for provider to learn about initiating PrEP, n (%)
Somewhat to strongly agree	29 (67.4)
Somewhat disagree to neither agree nor disagree	0 (0.0)
Not applicable/did not use	14 (32.6)
PrEP order set was helpful to guide provider through ordering PrEP, n (%)
Somewhat to strongly agree	19 (44.2)
Somewhat disagree to neither agree nor disagree	5 (11.6)
Not applicable/did not use	19 (44.2)
Option to refer patient helped provider to recommend PrEP, n (%)
Somewhat to strongly agree	19 (44.2)
Somewhat disagree to neither agree nor disagree	5 (11.6)
Not applicable/did not use	19 (44.2)
Provider liked convenience of having PrEP education module sent to EHR inbox, n (%)
Somewhat to strongly agree	26 (60.5)
Somewhat disagree to neither agree nor disagree	6 (14.0)
Not applicable/did not use	11 (25.6)
PrEP education module sent to EHR inbox was useful to teach provider about prescribing PrEP, n (%)
Somewhat to strongly agree	17 (39.5)
Somewhat disagree to neither agree nor disagree	8 (18.6)
Not applicable/did not use	18 (41.9)

Abbreviations: CDSS, clinical decision support system; EHR, electronic health record; PrEP, preexposure prophylaxis.

Table 5 Qualitative provider feedback on PrEP clinical decision support system (CDSS)

Theme	Subtheme	Details
Part of the PrEP CDSS providers found most helpful	PrEP awareness	Provider unaware PrEP existed prior to CDSS
		Understanding who would benefit from PrEP
		Awareness of need for further self-education about PrEP
	HIV test order alert	Reminder to discuss PrEP while ordering HIV/STI screening labs
	PrEP order set	Support to initiate PrEP with patients
		EHR support to ensure appropriate screening tests and labs ordered
		EHR support to order PrEP medication
	Referral order to PrEP provider	Desire to give patients better PrEP support through referral to PrEP experts
		Previous provider experience not knowing where patients could receive additional PrEP support
	Education module	Accessibility of module in EHR message inbox
		Flowchart with step-by-step approach to initiating PrEP
		PrEP follow-up guidelines
		PrEP case examples
		References for additional PrEP education
What providers plan to use from the PrEP CDSS	PrEP initiation	Knowledge to offer PrEP to patients who would benefit
		Order set assistance for medication and lab orders
		Referral to PrEP provider
	PrEP education	Review of education module
		Review and learn from CDSS PrEP order set recommendations
		Plan to continue PrEP self-education
Suggested improvements to make CDSS more effective	Alert and CDSS timing	Alert with lab order too late in visit so there was not enough time to discuss PrEP
		Alternative location/trigger in EHR for CDSS
		Remove alert from patients who would not benefit from PrEP
	Educational materials and resources	Easier accessibility to educational module for review/reference
		Patient-facing educational materials online
		Studies on PrEP efficacy and safety profile
		Guidance on PrEP and adolescent confidentiality
		Information on paying for PrEP

Abbreviations: CDSS, clinical decision support system; EHR, electronic health record; HIV, human immunodeficiency virus; PrEP, preexposure prophylaxis, STI, sexually transmitted infections.

The primary outcomes were changes in provider PrEP knowledge and changes in self-reported likelihood to prescribe and refer a patient for PrEP before and after exposure to the PrEP CDSS. Provider PrEP knowledge was self-reported on a five-point scale as described in [Tables 2] and [3].

Secondary outcomes were provider-reported utilization and utility of the CDSS, provider perceptions of barriers to recommending and/or prescribing PrEP, and provider feedback regarding how to improve the usability and usefulness of the CDSS. Providers were able to respond to these areas of the survey using a mix of Likert scales, selection from a given list of options, and prompts with free-text responses.

Statistical Analyses

McNemar's test was used to evaluate the primary outcomes of change in provider PrEP knowledge and provider likelihood to prescribe and refer a patient for PrEP. This was done by comparing the marginal frequencies of providers who reported being at least “moderately knowledgeable” (compared to “not at all” or “slightly knowledgeable”) and at least “somewhat likely” to prescribe or refer for PrEP (compared to “extremely unlikely,” “somewhat unlikely,” or “neither likely nor unlikely”), before and after exposure to the CDSS. Frequency analyses were conducted to describe participant demographic characteristics, PrEP prescribing experience prior to exposure to the CDSS, perceived utilization and utility of the CDSS, and provider-reported barriers to initiating PrEP.

A post-hoc sensitivity analysis was done limiting the analysis to pediatric primary care providers. This excluded adolescent medicine specialists and specialty care providers.

Qualitative free-text response data were analyzed using framework analysis[43] to organize responses into identified themes and subthemes. Qualitative findings were connected to elements from the quantitative analysis using the merge approach.[44]

Statistical analyses were done using Stata 15.1. This study was approved by the Stanford University Institutional Review Board. Data were analyzed from April 2022 to June 2022.

Results

Participant Characteristics

A total of 73 providers were invited to participate in the survey. Of those invited, 46 participants (63%) completed the survey. Three responses were excluded due to missing data on the primary outcomes. Among the 43 included participants ([Table 1]), mean age was 40.4 years. Participants most commonly identified as female (65%) and Asian/Pacific islander (47%) or white (35%). Most providers were physicians (91%) and 7% were nurse practitioners with a mean of 9.7 years in practice. A majority (63%) practiced in pediatric primary care, 21% practiced in specialty care, and 14% practiced in adolescent medicine. Prior to being exposed to the CDSS, 86% had never been prescribed PrEP.

Change in Provider Preexposure Prophylaxis Knowledge and Likelihood to Prescribe Preexposure Prophylaxis or Refer a Patient for Preexposure Prophylaxis

Among the total study sample, the proportion of providers who reported being “moderately” to “very knowledgeable” significantly increased from 28% before CDSS exposure to 61% after CDSS exposure (p = 0.0001; [Table 2]). The proportion of providers who were “somewhat” to “extremely likely” to prescribe PrEP to their patients significantly increased from 9% before using the CDSS to 51% after (p < 0.0001). The proportion of providers who were “somewhat” to “extremely likely” to refer a patient for PrEP significantly increased from 42% before CDSS exposure to 86% after (p < 0.0001).

When limiting the analysis to pediatric primary care providers, 27 responses were included in the analysis ([Table 3]). Of the 27 providers, 96% reported never having prescribed PrEP prior to using the CDSS. Among the primary care provider sample, the proportion of providers who reported being “moderately” to “very knowledgeable” significantly increased from 15% before CDSS exposure to 63% after (p = 0.0002). The proportion of providers who were “somewhat” to “extremely likely” to prescribe PrEP to their patients significantly increased from 0% before using the CDSS to 52% after (p = 0.0001). The proportion of providers who were “somewhat” to “extremely likely” to refer a patient for PrEP significantly increased from 26% before CDSS exposure to 89% after (p = 0.0001).

Provider Perceptions on Utility of the Clinical Decision Support System

Among the study sample, 67% of providers reported interacting with the PrEP CDSS, all of whom agreed that the CDSS was an effective educational tool to learn about prescribing PrEP ([Table 4]). The most common qualitative feedback from providers regarding what was most helpful from the CDSS was PrEP awareness ([Table 5]). Providers reported that they were unaware PrEP existed prior to seeing the CDSS alert. The CDSS gave them a beginning understanding of what PrEP was and who would benefit from PrEP and also brought awareness to the need for further self-education about PrEP.

When asked about the available CDSS options to prescribe and/or refer for PrEP, 44% of providers answered that the standardized order set was helpful to guide them through ordering PrEP, 44% did not use the order set, and 12% neither agreed nor disagreed that it was helpful. Qualitative feedback demonstrated that those who did use the order set found it helpful to initiate PrEP with their patients through guidance to order appropriate screening labs and medication. Similarly, 44% reported that the option to refer their patient directly to a PrEP provider helped them to recommend PrEP. Of the participants, 40% reported that they would prefer to refer their patients to a specialized PrEP provider instead of managing PrEP in their own practice. In their qualitative feedback, some providers elaborated on the desire to refer to PrEP experts who could offer patients more robust PrEP care and recounted experiencing past barriers to recommending PrEP due to not knowing where patients could receive PrEP support.

When asked about the option to send the PrEP educational module to their EHR inbox, 61% reported liking the convenience of sending themselves the PrEP module and 40% reported that the module was useful to learn about prescribing PrEP. Qualitative feedback demonstrated that providers appreciated the accessibility of the PrEP module in their EHR inbox. Providers reported that the step-by-step guideline for initiating PrEP, clear follow-up protocol, case study exercises, and references for additional PrEP education were some of the most helpful resources.

Participants also provided qualitative feedback regarding what they planned to continue using from the PrEP CDSS. Some providers reported that the PrEP CDSS gave them the knowledge and tools they needed to initiate PrEP and specifically planned to continue utilizing the standardized order set and the referral to a specialized PrEP provider for patients who would benefit from additional PrEP support. Other providers reported that they planned to review the PrEP education module, go through the PrEP order set to learn how to manage PrEP, and continue self-education about PrEP through the provided resources.

Lastly, providers suggested improvements through qualitative feedback that could make the PrEP CDSS more effective. Providers reported that they were oftentimes placing HIV test orders in the EHR at the end of their visits, at which point there was insufficient time to discuss PrEP after seeing the CDSS. Therefore, there were recommendations to consider alternative “triggers” for the CDSS alert to appear earlier in the visit workflow. One provider shared that the CDSS alert was disruptive when they were ordering routine STI screening for a patient that was not “part of a high-risk group.” Providers also expressed a desire for additional educational materials and resources including easier accessibility to the PrEP educational module for review and reference, more data on PrEP efficacy and safety, and direct access to patient-facing PrEP educational materials. Providers also wanted guidance in the CDSS regarding how to navigate PrEP initiation and maintenance in the setting of adolescent confidentiality concerns and how to navigate paying for PrEP taking into account insurance status and confidentiality needs. Quantitative survey data demonstrated that one-third (33%) of providers felt that concerns about adolescent confidentiality were notable barriers to prescribing or referring their patients for PrEP. One-third (33%) of providers also cited unfamiliarity with navigating paying for PrEP and concerns regarding insurance coverage as barriers to integrating PrEP into their practices.

Discussion

This study evaluated changes in pediatric provider knowledge of, likelihood to prescribe, and likelihood to refer a patient for PrEP after exposure to a PrEP CDSS. Among respondents, provider knowledge and likelihood to prescribe and refer a patient for PrEP all significantly increased after being exposed to the PrEP CDSS. Our findings demonstrate that a PrEP CDSS can be an effective tool to increase PrEP knowledge and willingness to initiate PrEP among pediatric providers.

Among participants, over two-thirds of providers interacted with the CDSS tools, and all who used at least one tool reported that it was effective to learn about initiating PrEP. These findings are important given previous literature, which demonstrated that provider knowledge was associated with PrEP initiation, prescribing, and future intention to prescribe.[9] [13] Prior studies have also found that providing basic PrEP education to providers had the potential to increase PrEP knowledge, confidence, and subsequent prescribing.[9] [13] [14] [15] [16] [17] [18] Our findings support previous PrEP literature and demonstrate that a PrEP CDSS may provide the needed education to increase provider PrEP knowledge and intention to initiate PrEP.[9] [14] [15] Our findings are also consistent with previous CDSS literature, which demonstrated that CDSS alerts directed at providers during order entry can be effective to improve appropriate medication prescribing and guideline compliance.[26] [27] [32]

We found a large gap in PrEP experience among pediatric providers, with 86% of participants having no prior PrEP prescribing experience. Multiple providers commented that they were unaware of PrEP prior to seeing the CDSS alert. This is consistent with prior studies, which have demonstrated a particularly large gap in provider knowledge and experience with PrEP in adolescent patients compared to older age groups.[7] [10] Previous literature has found that interventions that support first-time PrEP initiation among PrEP-inexperienced providers may have the greatest impact, as providers with experience had higher intention to prescribe PrEP in the future.[16] [40] It is important to provide both prescribing support and referral options, as PrEP-inexperienced providers learning about PrEP for the first time may not be comfortable immediately prescribing PrEP. This may be evidenced by our finding that a majority of providers did not use the PrEP order set or did not find it helpful potentially because they did not feel prepared to order the labs and medication. In these cases, providers may feel more comfortable initiating a referral for PrEP care. Almost half of participating providers reported that they preferred to refer their patient to a specialized PrEP provider. Therefore, a CDSS that offers a standardized PrEP order set and referral option can support a PrEP-inexperienced provider to initiate PrEP for the first time. Providers in this study reported an increased intention to continue initiating PrEP in the future using the CDSS order set and referral tools.

The magnitude of the increase in PrEP knowledge and likelihood to prescribe or refer a patient for PrEP was even greater when limiting the analysis to pediatric primary care providers, who were a key target population for the PrEP CDSS intervention. Primary care is oftentimes the ideal setting for PrEP provision; however, 96% of the primary care providers in our study had never prescribed PrEP. Previous studies have found that most primary care providers are unfamiliar with PrEP guidelines, unable to identify candidates for PrEP, and uncomfortable initiating PrEP.[7] [9] [10] [11] Our findings demonstrate that a PrEP CDSS may address these barriers. Compared to before seeing the CDSS, there were significant increases in providers who considered themselves knowledgeable about PrEP from 15 to 63%, who were likely to prescribe PrEP from none to 52%, and who were likely to refer a patient for PrEP from 26 to 89% after being exposed to the CDSS. CDSS tools may also be particularly useful in primary care, where providers have a wide scope of conditions they must manage, to reduce the burden of what providers must memorize and support efficient and guideline-based care.

Our findings are particularly important given the relatively high rates of HIV among adolescents and young adults, low rates of provider intention to prescribe, and low rates of PrEP provision compared to other age groups.[7] [10] [21] [45] It is necessary to empower providers to bring up PrEP with their patients to remove the burden of initiating PrEP conversations from youth who may not volunteer information about their sexual practices, may not be aware of PrEP, and may not be comfortable asking for PrEP.[9] [46] A prior study of veterans affairs patients demonstrated that 94% of PrEP conversations were patient initiated,[47] and it is likely that youth are less willing to bring up PrEP than adult patients. There is currently an undue burden on patients to ask for PrEP when these conversations should instead be initiated and normalized by their providers. The American Academy of Pediatrics recommends that pediatric providers routinely offer PrEP to all youth at risk for HIV.[48] Tools such as a PrEP CDSS can support and empower providers to have PrEP conversations that they otherwise may not have due to the lack of knowledge and confidence.

We also solicited provider suggestions regarding how to make the PrEP CDSS more effective. There were requests to make the educational module and patient-facing materials easily accessible online and not only through the CDSS within the EHR. There were also requests to provide more data on PrEP safety and efficacy, which may alleviate concerns regarding prescribing a new medication for providers unfamiliar with PrEP.[18] Providers also wanted to have additional guidance on navigating adolescent confidentiality and paying for PrEP in the CDSS. This is an important suggestion to consider given previous evidence demonstrating that concerns about managing adolescent confidentiality and consent as well as cost and insurance factors were associated with lower intention to prescribe PrEP.[7] [10]

It is necessary to address pain points reported by providers, as the effectiveness, acceptability, and ultimately adoption of CDSS tools can be hindered by issues with usability, disruptions to provider workflows, and alert fatigue.[22] [23] [24] [25] Some providers reported that the alert attached to an HIV test order was too late in the visit workflow. Providers oftentimes place orders at the end of or after a patient visit, at which point the opportunity to have an informed discussion about PrEP has passed. Therefore, an alternate trigger or location within the EHR for the CDSS should be considered. One suggested option has been to integrate a CDSS into progress notes, which seem to be the center point of provider interaction with the EHR during clinic visits.[49] This option would limit interruptions to visit workflows and allows providers to easily refer back to CDSS tools at any point.[49]

A provider also shared that the CDSS alert was disruptive when ordering routine STI screening for a patient that they did not think would benefit from PrEP. One potential consideration is whether the CDSS alert should be interruptive—which forces provider response in the middle of a workflow and may increase alert fatigue—or noninterruptive—which does not force immediate response but has been shown to be less successful at changing provider behavior.[25] [50] [51] [52] This provider's feedback also suggests that the CDSS could potentially improve its sensitivity by utilizing available patient data to better predict if a patient would benefit from PrEP at that visit. A hybrid design to consider may be a noninterruptive PrEP CDSS that is available on-demand throughout the visit workflow paired with well-timed and highly applicable interruptive alerts for patients who would most benefit.[50] To optimize the effectiveness of this and any CDSS, there needs to be continuous improvement and feedback from end-users to upkeep a tool that is simple, quick, relevant, and adaptive; tailored to provider workflows; and supports evidence-based practice changes.[53] [54]

Future directions to follow-up on this study include refining the CDSS based on provider feedback and rigorously testing the updates. This includes creating a CDSS version that is noninterruptive and appears earlier in the visit workflow that can be randomized within the EHR and evaluated for efficacy in increasing PrEP initiations and for usability and acceptability among providers. Further studies evaluating prescribing behavior among exposed providers and studies with greater follow-up should be done to determine the longer-term effects of the CDSS on provider behavior.

There are several limitations to our study that warrant consideration. There may be response bias where participants who had the most positive or most negative experiences with the PrEP CDSS may have been more likely to respond. While we evaluated knowledge and likelihood to initiate PrEP, our study did not follow actual PrEP prescribing behavior among participants, although a previous publication studying this intervention found an increase in PrEP prescriptions after implementation of the PrEP CDSS.[41] We only solicited provider feedback at one point in time and therefore are not able to evaluate longer-term provider behavior after initial exposure to the CDSS. This study was conducted at a single institution, which may limit the generalizability of our findings.

Conclusion

This study suggests that an interruptive PrEP CDSS attached to HIV test orders can be an effective tool to increase PrEP knowledge and likelihood to initiate PrEP among pediatric providers. Continual improvement and design of the PrEP CDSS based on provider feedback is required to optimize the usability, acceptability, and ultimately adoption and effectiveness of the tool. A highly usable PrEP CDSS may be a powerful tool to close the gap in PrEP access and uptake among the youth population.

Clinical Relevance Statement

There is a large knowledge and experience gap when it comes to HIV PrEP among providers who care for the youth population. This contributes to adolescents and young adults having the greatest unmet PrEP need compared to all other age groups, resulting in youth being exposed to HIV without available protection and ultimately preventable new HIV infections. A PrEP CDSS in the EHR can be effective in increasing PrEP knowledge and likelihood to initiate PrEP among pediatric and adolescent providers.

Multiple Choice Questions

1. What percentage of pediatric providers in this study had experience prescribing PrEP prior to exposure to the PrEP CDSS?

   • Greater than 75%

   • Between 50 to 75%

   • Between 25 to 50%

   • Less than 25%

   Correct Answer: The correct answer is option d. Prior to being exposed to the CDSS, 86% of pediatric providers had never prescribed PrEP. When limiting the analysis to pediatric primary care providers, 96% reported never having prescribed PrEP prior to using the CDSS. In both groups, far less than 25% of providers had experience prescribing PrEP prior to exposure to the PrEP CDSS.

2. Which of the following outcomes were found to have a statistically significant increase after provider exposure to the PrEP CDSS compared to before exposure to the PrEP CDSS?

   • Provider PrEP knowledge

   • Provider likelihood to prescribe PrEP

   • Provider likelihood to refer a patient for PrEP

   • All of the above

   Correct Answer: The correct answer is option d. Among the total study sample, the proportion of providers who reported being moderately to very knowledgeable significantly increased from 28% before CDSS exposure to 61% after CDSS exposure (p = 0.0001). The proportion of providers who were “somewhat” to “extremely likely” to prescribe PrEP to their patients significantly increased from 9% before using the CDSS to 51% after using the CDSS (p < 0.0001). The proportion of providers who were “somewhat” to “extremely likely” to refer a patient to PrEP significantly increased from 42% before CDSS exposure to 86% after CDSS exposure (p < 0.0001).

• References

• 1 Centers for Disease Control and Prevention. HIV Surveillance Report, 2020; vol. 33. Published May 2022. Accessed July 9, 2022 at: https://www.cdc.gov/hiv/library/reports/hiv-surveillance.html
• 2 Preexposure Prophylaxis (PrEP) | HIV Risk and Prevention | HIV/AIDS | CDC. Accessed February 6, 2021 at: https://www.cdc.gov/hiv/risk/prep/index.html
• 3 Centers for Disease Control and Prevention. Core indicators for monitoring the Ending the HIV Epidemic initiative: National HIV Surveillance System Data Reported through December 2021; and Preexposure Prophylaxis (PrEP) Data Reported through September 2021. HIV Surveillance Data Tables 2022;3 (No. 1). . Published May 2022. Accessed July 9, 2022 at: https://www.cdc.gov/hiv/library/reports/surveillance-data-tables/index.html
• 4 Huang YLA, Zhu W, Wiener J, Kourtis AP, Hall HI, Hoover KW. Impact of coronavirus disease 2019 (COVID-19) on human immunodeficiency virus (HIV) preexposure prophylaxis prescriptions in the United States—a time-series analysis. Clin Infect Dis 2022; 75 (01) e1020-e1027
• 5 Henny KD, Duke CC, Geter A. et al. HIV-related training and correlates of knowledge, HIV screening and prescribing of nPEP and PrEP among primary care providers in southeast United States, 2017. AIDS Behav 2019; 23 (11) 2926-2935
• 6 Liu A, Cohen S, Follansbee S. et al. Early experiences implementing preexposure prophylaxis (PrEP) for HIV prevention in San Francisco. PLoS Med 2014; 11 (03) e1001613
• 7 Mullins TLK, Idoine CR, Zimet GD, Kahn JA. Primary care physician attitudes and intentions toward the use of HIV preexposure prophylaxis in adolescents in one metropolitan region. J Adolesc Health 2019; 64 (05) 581-588
• 8 Zhang C, McMahon J, Fiscella K. et al. HIV preexposure prophylaxis implementation cascade among health care professionals in the United States: Implications from a systematic review and meta-analysis. AIDS Patient Care STDS 2019; 33 (12) 507-527
• 9 Moore E, Kelly SG, Alexander L. et al. Tennessee healthcare provider practices, attitudes, and knowledge around HIV preexposure prophylaxis. J Prim Care Community Health 2020; 11: 2150132720984416
• 10 Pina P, Taggart T, Sanchez Acosta M, Eweka I, Muñoz-Laboy M, Albritton T. Provider comfort with prescribing HIV preexposure prophylaxis to adolescents. AIDS Patient Care STDS 2021; 35 (10) 411-417
• 11 Pleuhs B, Quinn KG, Walsh JL, Petroll AE, John SA. Health care provider barriers to HIV preexposure prophylaxis in the United States: a systematic review. AIDS Patient Care STDS 2020; 34 (03) 111-123
• 12 Mullins TLK, Zimet G, Lally M, Xu J, Thornton S, Kahn JA. HIV care providers' intentions to prescribe and actual prescription of preexposure prophylaxis to at-risk adolescents and adults. AIDS Patient Care STDS 2017; 31 (12) 504-516
• 13 Blumenthal J, Jain S, Krakower D. et al; CCTG 598 Team. Knowledge is power! Increased provider knowledge scores regarding preexposure prophylaxis (PrEP) are associated with higher rates of PrEP prescription and future intent to prescribe PrEP. AIDS Behav 2015; 19 (05) 802-810
• 14 Blackstock OJ, Moore BA, Berkenblit GV. et al. A cross-sectional online survey of HIV preexposure prophylaxis adoption among primary care physicians. J Gen Intern Med 2017; 32 (01) 62-70
• 15 Clement ME, Seidelman J, Wu J. et al. An educational initiative in response to identified PrEP prescribing needs among PCPs in the Southern U.S. AIDS Care 2018; 30 (05) 650-655
• 16 Tripathi A, Ogbuanu C, Monger M, Gibson JJ, Duffus WA. Preexposure prophylaxis for HIV infection: healthcare providers' knowledge, perception, and willingness to adopt future implementation in the southern US. South Med J 2012; 105 (04) 199-206
• 17 Bacon O, Gonzalez R, Andrew E. et al. Brief report: informing strategies to build PrEP capacity among San Francisco Bay area clinicians. J Acquir Immune Defic Syndr 2017; 74 (02) 175-179
• 18 Turner L, Roepke A, Wardell E, Teitelman AM. Do you PrEP? A review of primary care provider knowledge of PrEP and attitudes on prescribing PrEP. J Assoc Nurses AIDS Care 2018; 29 (01) 83-92
• 19 Sales JM, Cwiak C, Haddad LB. et al. Brief Report: impact of PrEP training for family planning providers on HIV prevention counseling and patient interest in PrEP in Atlanta, Georgia. J Acquir Immune Defic Syndr 2019; 81 (04) 414-418
• 20 Pinto RM, Witte SS, Filippone P, Choi CJ, Wall M. Interprofessional collaboration and on-the-job training improve access to hiv testing, HIV primary care, and preexposure prophylaxis (PREP). AIDS Educ Prev 2018; 30 (06) 474-489
• 21 Harris NS, Johnson AS, Huang YA. et al. Vital signs: status of human immunodeficiency virus testing, viral suppression, and HIV preexposure prophylaxis - United States, 2013-2018. MMWR Morb Mortal Wkly Rep 2019; 68 (48) 1117-1123
• 22 Backman R, Bayliss S, Moore D, Litchfield I. Clinical reminder alert fatigue in healthcare: a systematic literature review protocol using qualitative evidence. Syst Rev 2017; 6 (01) 255
• 23 Nanji KC, Seger DL, Slight SP. et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc 2018; 25 (05) 476-481
• 24 Payne TH, Hines LE, Chan RC. et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc 2015; 22 (06) 1243-1250
• 25 Horsky J, Phansalkar S, Desai A, Bell D, Middleton B. Design of decision support interventions for medication prescribing. Int J Med Inform 2013; 82 (06) 492-503
• 26 Becker ML, Baypinar F, Pereboom M. et al. The effect of medication related clinical decision support at the time of physician order entry. Int J Clin Pharm 2021; 43 (01) 137-143
• 27 May A, Hester A, Quairoli K, Wong JR, Kandiah S. Impact of clinical decision support on azithromycin prescribing in primary care clinics. J Gen Intern Med 2021; 36 (08) 2267-2273
• 28 Ostropolets A, Zhang L, Hripcsak G. A scoping review of clinical decision support tools that generate new knowledge to support decision making in real time. J Am Med Inform Assoc 2020; 27 (12) 1968-1976
• 29 Oluoch T, Santas X, Kwaro D. et al. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review. Int J Med Inform 2012; 81 (10) e83-e92
• 30 Gibbs KD, Shi Y, Sanders N. et al. Evaluation of a sepsis alert in the pediatric acute care setting. Appl Clin Inform 2021; 12 (03) 469-478
• 31 Friebe MP, LeGrand JR, Shepherd BE, Breeden EA, Nelson SD. Reducing inappropriate outpatient medication prescribing in older adults across electronic health record systems. Appl Clin Inform 2020; 11 (05) 865-872
• 32 Shahmoradi L, Safdari R, Ahmadi H, Zahmatkeshan M. Clinical decision support systems-based interventions to improve medication outcomes: a systematic literature review on features and effects. Med J Islam Repub Iran 2021; 35: 27
• 33 Schedlbauer A, Prasad V, Mulvaney C. et al. What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior?. J Am Med Inform Assoc 2009; 16 (04) 531-538
• 34 Hansen MJ, Carson PJ, Leedahl DD, Leedahl ND. Failure of a best practice alert to reduce antibiotic prescribing rates for acute sinusitis across an integrated health system in the midwest. J Manag Care Spec Pharm 2018; 24 (02) 154-159
• 35 Devries J, Rafie S, Polston G. Implementing an overdose education and naloxone distribution program in a health system. J Am Pharm Assoc (Wash DC) 2017; 57 (2S): S154-S160
• 36 Malte CA, Berger D, Saxon AJ. et al. Electronic medical record alert associated with reduced opioid and benzodiazepine coprescribing in high-risk veteran patients. Med Care 2018; 56 (02) 171-178
• 37 Strom BL, Schinnar R, Aberra F. et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med 2010; 170 (17) 1578-1583
• 38 Jaspers MWM, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011; 18 (03) 327-334
• 39 Kwan JL, Lo L, Ferguson J. et al. Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials. BMJ 2020; 370: m3216
• 40 van den Berg P, Powell VE, Wilson IB, Klompas M, Mayer K, Krakower DS. Primary care providers' perspectives on using automated HIV risk prediction models to identify potential candidates for preexposure prophylaxis. AIDS Behav 2021; 25 (11) 3651-3657
• 41 Chan CT, Vo M, Carlson J, Lee T, Chang M, Hart-Cooper G. Pediatric provider utilization of a clinical decision support alert and association with HIV preexposure prophylaxis prescription rates. Appl Clin Inform 2022; 13 (01) 30-36
• 42 Centers for Disease Control and Prevention (CDC). Recommended Training Effectiveness Questions for Postcourse Evaluations: User Guide. CDC; 2019
• 43 Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13 (01) 117
• 44 Moseholm E, Fetters MD. Conceptual models to guide integration during analysis in convergent mixed methods studies. Methodol Innov 2017; 10 (02)
• 45 Dailey A, Gant Z, Johnson S. et al. HIV Surveillance Report 2018 (Updated).; 2018
• 46 Owens DK, Davidson KW, Krist AH. et al; US Preventive Services Task Force. Screening for HIV infection: US preventive services task force recommendation statement. JAMA 2019; 321 (23) 2326-2336
• 47 Skolnik AA, Bokhour BG, Gifford AL, Wilson BM, Van Epps P. Roadblocks to PrEP: what medical records reveal about access to HIV preexposure prophylaxis. J Gen Intern Med 2020; 35 (03) 832-838
• 48 Hsu KKC, Rakhmanina NY, Chadwick EG. et al. Adolescents and young adults: the pediatrician's role in HIV testing and pre- and postexposure HIV prophylaxis. Pediatrics 2022; 149 (01) e2021055207
• 49 Richardson S, Feldstein D, McGinn T. et al. Live usability testing of two complex clinical decision support tools: observational study. JMIR Human Factors 2019; 6 (02) e12471
• 50 Blecker S, Pandya R, Stork S. et al. Interruptive versus noninterruptive clinical decision support: usability study. JMIR Human Factors 2019; 6 (02) e12469
• 51 Patterson ES, DiLoreto GN, Vanam R, Hade E, Hebert C. Enhancing usefulness and usability of a clinical decision support prototype for antibiotic stewardship. Proc Int Symp Hum Factors Ergon Healthc 2020; 9 (01) 61-65
• 52 Phansalkar S, van der Sijs H, Tucker AD. et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (03) 489-493
• 53 Salwei ME, Hoonakker P, Carayon P, Wiegmann D, Pulia M, Patterson BW. Usability of a human factors-based clinical decision support in the emergency department: lessons learned for design and implementation. Hum Factors 2022; 187208221078625
• 54 Bates DW, Kuperman GJ, Wang S. et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530