Development and use of a clinical decision support tool for behavioral health screening in primary care clinics

Keywords

Electronic health records - clinical decision support systems - behavioral medicine - depression - SBIRT

1. Background and Significance

The need to identify and respond to primary care (PC) patients who require behavioral interventions to optimize health function is becoming widely accepted.[[1]–[3]] Recently, the American Academy of Family Physicians endorsed amending the Joint Principles of the Patient Centered Medical Home (PCMH) to include behavioral services as a core element, and emphasized the importance of integrated behavioral health care [[1]]. Specific components of the PCMH behavioral services include depression, alcohol use, and substance abuse. This position statement is supported by three trials showing benefit from behavioral treatment initiation in the primary care setting [[4]–[6]]. Although the acronym SBIRT (screening, brief intervention, and referral for treatment) was initially used in the context of substance abuse interventions, the framework can be applied to other behavioral health conditions.

For purposes of our study, we expanded the definition and applied SBIRT to depression, anxiety, and prescription misuse. Screening usually involves a validated, patient questionnaire; often the tool begins with a limited number of initial questions with high sensitivity, and if the responses are positive followed by several more questions to increase specificity. The brief intervention consists of several minutes of counseling at the time of the screening. Referral for treatment involves arranging for in-depth evaluation and management services with a licensed behavioral health specialist.

In response to increased attention to the provision of behavioral services within primary care and its effectiveness in responding to the high levels of need in the primary care population, there have been multiple regulatory and measurement changes in this domain. The Centers for Medicare and Medicaid Services (CMS) now reimburses depression and substance use screening, and such screenings are included as preventative services in the Affordable Care Act. CMS also included depression screening as 1 of the 33 mandatory measures for Accountable Care Organizations (ACO Measure 18; NQF #0418) [[7]]. Starting with the 2011 standards, the National Committee for Quality Assurance requires screening for depression as part of certification for Patient-Centered Medical Homes [[8], [9]]. Based on demonstrated improvement in clinical outcomes, the United States Preventative Services Task Force (USPSTF) has recommended screening and behavioral intervention for alcohol abuse in all adults since 2013. In 2016, they began recommending screening for depression in all patients ages 12 years and older in practices that can provide services to address positive screens (both recommendations Level of evidence grade B) [[10], [11]]. Depression screening and treatment resulted in improved remission and/or response rates from 17% to 87% based on 9 studies [[10]]. For alcohol use, reviews found that brief intervention (BI) of 10–15 minutes was the most effective; heavy drinking decreased by approximately 12% on average, and consumption went down by an average of 3.6 drinks per week [[11]].

Historically, rates for behavioral health screening have been quite low despite these recommendations. These low screening rates are due, in part, to the lack of training on the need for screening, the inaccessibility of screening tools at the point of care, and poor integration of screening into clinical workflows and EHRs. A standardized method of primary care behavioral data collection and real time decision support available could assist PC decision-making, but efforts to engage in such activities have been challenging. Kessler [[3], [4]] has identified at least 3 critical dimensions necessary for screening efforts to have impact:

1. adequate resources to interpret positive screens;

2. appropriate available interventions;

3. the need to pay attention to potential negative effects such as patient reaction or refusal, staff resistance to the process, lack of clearly thought out implementation strategies.

Clinical decisions support (CDS) tools in the electronic health record (EHR), if properly designed and implemented, could address many of these concerns. Health information technology (HIT) can support systematic collection of patient data, the transformation of those data into clinically-relevant information through programmed algorithms, and the generation and presentation of knowledge in the form of evidence-driven, patient-specific treatment recommendations. Such tools can be scaled from small clinics to large institutions to create a reliable, high-value, patient-centric care model. Specifically, our goal was to develop CDS tools that would 1) semi-automate the interpretation of screening to reduce provider cognitive load; 2) triage patients with the greatest need for intervention or referral in order to improve use of limited treatment resources; and 3) create a workflow that overcame technical and social barriers to change.

Such CDS tools have demonstrated high reliability in improving preventative care services [[12]–[14]] and in the diagnosis and management of BH diagnoses [[15]]. Other reviews and expert panels have attempted to identify the design qualities of CDS which are more likely to lead to widespread adoption and improvement of clinical quality measures [[16], [17]]. These recommendations have been summarized in the “CDS Five Rights” [[18]]:

1. Right information – applying evidence-based guidelines, tailored to the patient based on demographics, disease status, goals of care, or other factors

2. Right person – delivering the alert to the right person in the care team (medical assistant, physician, care coordinator, etc.)

3. Right CDS intervention format – recommending an appropriate intervention (e.g., flowsheet, order set, medication, lab test, or reference to a clinical guideline)

4. Right channel – delivering the recommendation via the best modality (e.g., in the EHR during an office visit or through communication in a patient portal)

5. Right time in the workflow – promoting the support at the right time in the sequence of clinical care (e.g., at front desk check-in, during the rooming process when vital signs are entered, during medication order entry, on closing the chart)

In addition to the CDS design principles, the team recognized that a solution required a multidisciplinary team including technical and clinical stakeholders [[19]].

Most commercial EHRs can be configured to include behavioral health screening questionnaires. Some EHR vendors, such as Epic (Verona, WI), include screening tools for depression and other conditions in their software. Despite the ubiquity of the screening tools, to the authors’ knowledge there are no well-designed reports published on the successful use of computer-based decision support integrated into clinic workflows.

2. Objectives

In response to the lack of integration of behavioral health screening and CDS into the EHR, a team of primary care and BH providers engaged with informaticians to fill this gap. We describe the processes and outcomes of building this clinical decision support (CDS) in the electronic health record (EHR). The first objective was to collect, score, and present behavioral health (BH) risk information to providers at the point of care in the exam room. We identified depression and alcohol use for SBIRT. In addition, due to increased local and national concerns, we specifically targeted use of medications in a fashion other than specified in the prescription. The second objective was to study outcomes from encounters using the CDS, compared to encounters when the BH screening tools were not used. The project was conducted as a pragmatic, applied clinical informatics improvement project.

3. Methods

The principal methods for this project included 1) gathering stakeholder consensus on clinical goals to address current gaps; 2) designing a future-state workflow and EHR tools for performing SBIRT processes; 3) implementation of the workflow and tools into PC clinics; and 4) the extraction and analysis of the use of these tools in the clinics. We did not attempt to enforce use of the tools or to randomize the implementation.

3.1 Setting

We implemented the new tools in 5 ambulatory primary care clinics affiliated with the University of Vermont Medical Center (Burlington, VT), an academic, tertiary care integrated delivery network using a shared instance of Epic EHR version 2010 (Verona, WI). Four of the clinics were with the Department of Family Medicine, the last 1 with the Department of Primary Care Internal Medicine. Each site had 6–9 providers, approximately half physicians with the remaining staff nurse practitioners and physician assistants. Nursing staff consisted of a mix of Registered Nurses (RN), Medical Assistants (MA), and Licensed Practical Nurses (LPN).

The clinics had been doing limited BH screenings, mostly on paper, for several years. Clinically, the goal was to screen all patients annually, but screening was done ad hoc with no systems to ensure it was done routinely. The content of the screening mostly focused on depression with the 9-itemPatient Health Questionnaire (PHQ), although there was substantial variability between and within clinic sites. Prior to this project in 2011, the EHR did not contain any CDS for BH, either health maintenance reminders or flowsheets for documenting BH screening. The BH tools for this study were built in 2011, and the 866 interventions for this study took place during 2011 and 2012.

3.2 Behavioral measurement selection

The first part of the project was the selection of BH measurement tools. We assembled an interdisciplinary team of 2 PC clinic medical directors, 2 PC informaticians, a PC behaviorist, and an RN clinic manager. The team agreed on screening in 3 clinical domains based on USPSTF recommendations: depression, alcohol use, and substance abuse. The general approach was to look for a 1- or 2-question initial screen with high negative predictive value. A positive screen would be followed up by a longer but more specific set of questions. In addition, there was clinical interest in following up a positive depression initial screen with a more detailed test for anxiety since the 2 conditions are frequently co-occurring [[20]] and have different treatments.

The team selected 4 frequently used, primary care focused, patient self-report measures:

1. For depression, the team chose the PHQ with the first 2 questions (PHQ-2) as the initial screen. If the PHQ-2 score was positive (> 3), it was followed with the longer 9-question version (PHQ-9) and the first 2 questions of the General Anxiety Disorder screen (GAD-2). A positive GAD-2 (> 3) was followed with the 7-question GAD-7 [[21]–[23]].

2. For alcohol use screening, the team selected the Alcohol Use Disorder Identification Test, with the first question (AUDIT-1) as the initial screen. We chose the AUDIT-1 in order to keep the total initial screen to only 4 questions for pragmatic reasons, recognizing the lower sensitivity compared with the 3-question AUDIT-C [[24], [25]]. A positive AUDIT-1 (> 3) was followed by the 10-item tool (AUDIT-10). (In retrospect, the validated single-question AUDIT-3 or the AUDIT-C would have been better.)

3. For substance abuse, the team selected the Prescription Misuse Question, a single item, Yes/No response question, derived from a similar validated questionnaire [[26]]. It is the most recently developed tool, without the published, validated psychometric history of the others, but targeted a clinical area of importance to stakeholders. A positive response (“Yes”) was followed by a 3-item test that included additional questions about substance misuse.

4. The overall BH screen process, consisting of the 4 initial questions, was defined as positive if any single domain item was positive (i.e. PHQ-2 > 3; or AUDIT-1 > 3; or Prescription Misuse = “Yes”). The 4 screening questions implemented were:

• PHQ-1: In the past 2 weeks, how often have you had little interest or pleasure in doing things? (0 = Not at all; 1 = Several days; 2 = More than half the days; 3 = Nearly every day.)

• PHQ-2: In the past 2 weeks, how often have you felt down, depressed, or hopeless? (0 = Not at all; 1 = Several days; 2 = More than half the days; 3 = Nearly every day.)

• AUDIT-1: How often do you have a drink containing alcohol? (0 = Never; 1 = Monthly or less; 2 = 2 to 4 times a month; 3 = 2 to 3 times a week; 4 = 4 or more times a week.)

• Prescription Misuse: Have you ever taken a prescription medicine other than as it was prescribed? (No; Yes).

3.3 Workflow design

The goal of our new BH workflow was synchronous collection of patient-reported outcomes (PRO) at the time of the visit, in the exam room, with minimal disruption to rooming staff and providers. In our final process, we eliminated paper responses in order to avoid transcription. Instead, rooming staff (MA, RN, LPN) were prompted to administer the screening verbally, reading the questions from the flowsheet to the patient, and recording the responses in the EHR flowsheet in real time before exiting the exam room. The goal was to keep providers from being involved in the collection or data-entry portions of the screening process and to present providers with summary scores and clinical recommendations for all three domains measured while in the exam room. Providers did not need to see responses to individual screening questions, but the data needed to be available for review if indicated.

The workflow was based on the need for screening in all 3 clinical domains regardless of whether or not the patient had previously been diagnose with a BH condition. Although not strictly screening anymore, clinical stakeholders felt that the screening tools would be useful in helping managed diagnosed but undertreated BH conditions.

3.4 CDS design, development, and implementation

The team discussed the desired goals for the tools with additional stakeholders, including rooming staff, providers, office supervisors, and behavioral health clinicians. They then used the 5 rights of CDS design to match the clinical use case and desired workflows with EHR technical tools. At the time of development there was no regulatory or institutional mandate for adoption of these BH screening CDS tools, so clinics used limited trainings that included an email with screenshots and a brief introduction to the tools at department meetings. There was also some uptake in diffusion from early adopters to peers in clinic, although the dissemination was neither part of the design nor evaluation.

Part of the desired EHR tools included the ability to track when the most recent screening occurred for an individual patient, and to create alerts prompting nursing staff to screen again if the patient was in clinic more than a year after the last screening date.

3.5 Plan-Do-Study-Act improvement iterations

We built an initial set of tools in the EHR and conducted a limited pilot with 4 providers in 2 clinics for 2 months. This early test indicated problems with the flowsheet design and decision support algorithms. Rather than continuing to test changes in the EHR, which is cumbersome to reconfigure, we reverted back to paper for an additional iteration. We deployed a modified paper prototype questionnaire and workflow with the same test users. This version of the tools passed usability testing, and we implemented the final CDS alerts and flowsheets in the EHR to all users in the PC clinics. Overall, this process involved 4 iterations over the course of 22 months. The final product, including workflows and EHR tools, is described in the Results section.

We did not formally evaluate the prototype solutions or the impact of our PDSA process on clinics.

3.6 Preliminary evaluation

The research team then took advantage of efforts by early clinical adopters to generate an observational design extract of a convenience sample of 866 encounters in which staff administered the 4-question BH screening CDS tool (intervention group). We also wanted to compare the outcomes of these intervention screening encounters with visits in which the BH screening tools were not used. We developed 3 groups of encounters to review:

1. Screening Encounters – This group included all 866 encounters in which BH screening happened during the visit. From these encounters, we created a list of patients who received the BH screening intervention at least once during the study period (Jan 2011 – Dec 2012).

2. Non-screening Encounters – From the list of patients who had BH screening with CDS tools done during the study period (Screening Encounters cohort), we identified an additional 850 encounters who did not have BH screening (Jan 2010 – Apr 2011)

3. Control Encounters – By excluding the screening encounter cohort, we created a list of 850 control patients who had been seen in the same clinics during the same time period as the intervention encounters and never had BH screening completed. From this list, we identified 1 control encounter per patient (Jan 2011 – Dec 2012).

Because of the pragmatic nature of this quality improvement project, we made no efforts to randomize at the patient, provider, or clinic level. We did not require that clinic staff perform BH screening or use the tools. All clinicians received similar access to the tools.

3.7 Aggregate data extraction and analysis

We extracted data from the Epic Clarity database using Microsoft SQL. We looked at all encounters in the Departments of Family Medicine and Primary Care Internal Medicine starting from the date the tools were built in the EHR production environment. The data were de-identified, removing all protected health information (PHI). The file was saved as a CSV for analysis using a set of scripts written in the Ruby programming language. Additional views of the data and graphics were generated in Microsoft Excel 2007.

We defined brief intervention (BI) and referral for treatment (RT) based on information that could readily be queried from the EHR database. BI was defined as occurring when the encounter provider 1) added a psychiatric diagnosis to the encounter diagnosis list; or 2) updated or added a new psychiatric diagnosis to the problem list; or 3) prescribed a new class of psychiatric medication. RT was defined as the encounter provider placing an electronic order for referral to a psychiatrist, a psychologist, or a case manager. Each of the BI and RT actions was tracked reliably in the EHR and could be extracted for analysis. These definitions are based on the assumption that a provider discussed BH issues with a patient during the encounter, but we did not attempt to determine the quality or duration of the counseling. We also did not attempt to measure compliance with medication changes or completion of referrals.

4. Results

The research question generated two aims:

1. Assess the feasibility of development and implementation of the BH screening workflow and associated CDS tools for BH screening; and

2. Conduct a preliminary evaluation of the CDS tools, including their use in clinics and the outcomes of encounters when the tools were used.

4.1 Development and implementation of BH screening workflow and CDS tools

The design process resulted in one new integrated workflow (►[Figure 1]) with 6 possible alerts (►[Table 1]) and one documentation flowsheet.

There were 3 types of patient-specific, synchronous alerts, or Best Practice Advisories (BPA): 1) 1 BPA to inform rooming staff (MA, LPN, or RN) that screening was overdue based on a health maintenance topic (►[Figure 2]); 2) 4 BPAs s to inform providers (MD, DO, NP, PA) of screening scores and recommended interventions (one BPA each for PHQ, GAD, AUDIT, and Prescription Misuse scores (►[Figure 3]); and 3) a BPA to prompt providers to enter an encounter diagnosis (V79.8 Screen-Mental Dis NEC), resetting the health maintenance screening trigger for another year. This last BPA and manual entry of an encounter diagnosis was a workaround because the Epic version 2010 provided no way to reset the HM for a year based on completion of the flowsheet.

Fig. 1 Clinical workflow and related decision support tools with algorithm logic (in green). Tools include health maintenance tool for tracking for periodic service Best Practice Advisory (BPA) alerts (in yellow) and cascading flowsheet for documenting BH screening (in blue). Tasks completed by rooming staff (in light gray) and providers (in orange) are tracked in the EHR for research analysis.

Fig. 2 Alert informs rooming staff (MA, LPN, MA) that BH screening is overdue based on health maintenance (HM) status. Entering heart rate triggers the evaluation of the HM status and fires the alert.

Fig. 3 Alerts evaluate the values in the BH screening flowsheets. When encounter provider opens the patient medical record, an alert makes recommendations based on the screening results.

Table 1

Taxonomy of CDS design to support BH screening in a primary care clinic.

CDS #1: Annual Behavioral Health Screening is Overdue
CDS Functional Category	CDS Taxonomy	Follow-on user actions
Trigger	Act	Rooming staff (MA, LPN, RN) enters heart rate in vital signs section of EHR
Input Data	Risk category/ Assessment	Health Maintenance status for BH screening is “Overdue”
	Care experience	Encounter providers in pilot study (initial) or in Family Medicine/ Primary Care Internal Medicine
	Care experience	Patient age ≥18 years
Interventions	Display	Pop-up alert for staff that BH screening is “Overdue”
Action Steps	Request	Link to documentation flowsheet for BH screening
	Acknowledge	Bypass alert (no reason required)
CDS #2, 3, 4, 5: Behavioral Health Screen Summary Score and Guided Clinical Recommendation
(1 alert each for depression, anxiety, alcohol, prescription misuse screens)
CDS Functional Category	CDS Taxonomy	Follow-on user actions
Trigger	Act	Rooming staff (MA, LPN, RN) enters values in BH screening documentation flowsheets in EHR
Input Data	Risk category/Assessment	Summary scores of BH screening from flowsheets
	Care experience	Encounter providers in pilot study (initial) or in Family Medicine/ Primary Care Internal Medicine
Interventions	Display	Pop-up alert for Provider with score of BH screening
	Display	Recommendation for clinical intervention based on score of BH screening
Action Steps	Acknowledge	Receipt or Bypass alert (no reason required)
CDS #6: Behavioral Health Screening Completed – Reset Health Maintenance for 1 Year
CDS Functional Category	CDS Taxonomy	Follow-on user actions
Trigger	Act	Rooming staff (MA, LPN, RN) enters values in BH screening documentation flowsheets in EHR
Input Data	System characteristic	Any value exists in the BH screening flowsheet (not NULL)
Interventions	Display	Pop-up alert for Provider that BH screening was performed
	Display	Recommendation to add Encounter Diagnosis “V79.8 Screen-mental dis NEC” in order to reset Health Maintenance for 1 year
Action Steps	Request	Link to Order Set with Encounter Diagnosis code V79.8
	Acknowledge	Receipt or Bypass alert (no reason required)

The dynamic flowsheet with structured questions was developed to prompt and record patient-reported outcomes (►[Figure 4]). The flowsheet begins showing only the 4 initial screen questions discussed previously: PHQ-2, AUDIT-1, and a single Prescription Misuse Question. Embedded decision support rules evaluate responses to these initial questions (►[Figure 1]). As indicated based on cut-off scores, the flowsheet automatically adds additional rows for full screening only as needed. For example, the scores for first and second questions of the PHQ-2 were automatically summed and evaluated; if the total was >3, the EHR automatically opened the additional 7 questions of PHQ-9 and the first 2 questions of GAD. These flowsheet rows allow the patient responses to be collected in the EHR without transcription from paper. By limiting the initial user interface to 4 questions and hiding the full PHQ-9, AUDIT-10, and Prescription Misuse 3 follow-up questions, the rooming staff were not overwhelmed based on both their input in the design process and by their anecdotal report of using the tools in practice.

Fig. 4 Documentation flowsheet for captures responses to BH screening. Flowsheet starts with only 4 required questions. Embedded decision support algorithm cascades open additional questions only when initial entries are above designated cut-off values.

4.2 Evaluation of CDS tools

Although we did not formally evaluate user experience, anecdotal reviews from nursing staff were positive. Of the 866 screening encounters, there were no cases where screening was started using the EHR tools and not completed.

►[Table 2] summarizes the 866 screenings performed subsequent to implementation in the production EHR environment. There were 8 encounters in the dataset where the screening was recorded twice in the same visit. In most cases, the RN did an initial screening, and the provider later changed the responses based on additional conversation with the patient.

Table 2

Characteristics of clinicians using behavioral health screening CDS tools.

Credential of clinical staff performing screening
	n	%
MA	353	40.76
RN	274	31.64
LPN	216	24.94
MD, DO	22	2.54
Other	1	0.12
NP, PA	0	0.00
Total	866	100.00
Clinical setting of screening
Family Medicine	n	%
Clinic A	475	54.85
Clinic B	242	27.94
Clinic C	86	9.93
Clinic D	60	6.93
Subtotal	863	99.65
Prim Care Internal Medicine	n	%
Clinic E	3	0.35
Subtotal	3	0.35
Total	866	100.00

►[Table 3] includes the patient demographics for the intervention encounters and controls (same patients during encounters without screening; matched patients during encounters with screening). There were no substantial age or gender differences between screened patients and comparison patients, indicating that our comparison group was adequately matched.

Table 3

Characteristics of patients screened with behavioral health CDS tools compared with control patients not screened.

	Patients Screened						Matched Control Patients (Not Screened)
	Screen Positive		Screen Negative		Subtotal
Age (years)	n	%	n	%	n	%	n	%
< 18	1	0.12	2	0.23	3	0.35	0	0.00
18–44	67	7.74	209	24.13	276	31.87	456	53.65
45–64	67	7.74	264	30.48	331	38.22	257	30.24
65–74	28	3.23	157	18.13	185	21.36	76	8.94
>74	16	1.85	55	6.35	71	8.20	61	7.18
Total	179	20.67	687	79.33	866	100.00	850	100.00
Sex	n	%	n	%	n	%	n	%
Female	100	11.55	407	47	507	58.55	460	54.12
Male	79	9.12	280	32.22	359	41.45	390	45.88
Total	179	20.67	687	79.33	866	100.00	850	100.00

The overall positive screen rate was 21% (179/866) (►[Figure 5]). Depression screening with PHQ-2 accounted for almost 60% of positive screens. Alcohol screening with AUDTI-1 represented a quarter of the positive screens, while the prescription misuse question accounted for only 7% of the positive response. There were only 15 encounters (11%) with positive screens in 2 of the 3 domains, and no encounters included positive screens in all 3 domains.

Fig. 5 Distribution of positive screens for 179 patients using the behavioral health CDS tools.

Clinical characteristics (prior BH issues on problem list, encounter diagnosis, psychotropic medication prescription, and referral for BH treatment) and the outcome of the encounters (brief interventions and referral for treatment) are summarized in ►[Table 4]. In all cases, screening increased brief intervention and referral for treatment rates, regardless of the outcome of the screening (positive or negative).

Table 4

Clinical characteristics and outcomes for patients screened with behavioral health CDS tools, comparing screened with control group.

	Intervention Patients (Screening done in at least 1 visit)								Matched Control Patients (Not Screened)
	Encounters with Screening						Encounters without Screening
	Screen Positive		Screen Negative		Subtotal
	n	%	n	%	n	%	n	%	n	%
Total	179	20.67	687	79.33	866	100.00	795	100.00	850	100.00
Behavioral Health (BH) characteristics
Prior BH diagnosis on Problem List	114	63.69	309	44.98	423	48.85	354	44.53	311	36.59
Prior BH Encounter Diagnosis	124	69.27	340	49.49	464	53.58	363	45.66	355	41.76
Prior psychiatric medications prescribed	109	60.89	313	45.56	422	48.73	346	43.52	322	37.88
Prior referral for BH treatment	50	27.93	129	18.78	179	20.67	35	4.40	107	12.59
Brief Interventions
New/updated BH diagnosis on Problem List	43	24.02	60	8.73	103	11.89	49	6.16	63	7.41
BH Encounter Diagnosis billed	131	73.18	212	30.86	343	39.61	148	18.62	159	18.71
New psychiatric medication prescribed	35	19.55	26	3.78	61	7.04	14	1.76	13	1.53
Referral for Treatment
New referral to mental health services	39	21.79	48	6.99	87	10.05	9	1.13	36	4.24

4.3 Comparisons of screened patients with patients who never received screening

Screened patients were more likely to have BH issues documented in past encounters compared with control patients who never received screening: prior BH problem list entry (49% v 37%); prior BH encounter diagnosis (54% v 42%); prior psychotropic medication prescribed (49% v 38%); and prior referral for BH treatment (21% v 13%) (►[Table 4]). Although we are not able to calculate statistical significance, this suggests a bias toward screening patients with prior BH diagnoses.

BH brief interventions and referrals were almost twice as frequent during encounters when patients were screened compared with encounters for control patients who never received screening: BH-related problem list updates (12% v 7%); encounter diagnoses (40% v 19%); psychotropic medication changes (7% v 2%); and BH referrals (10% v 4%) (►[Table 4]). The increased rates of brief intervention and referral during encounters with screening could reflect higher rates of BH morbidity in screened patients and/or could be attributed to the screening process itself as causative.

4.4 Comparisons of screening and non-screening encounters for same patients

Screening encounters, whether positive or negative, were approximately twice as likely to result in brief interventions and referrals, compared with encounters for the same patient during encounters when screening did not occur (►[Table 4]). Problem list updates related to BH happened in 12% of screening encounters compared with 6% of non-screening encounters. BH encounter diagnoses were used in 40% of screening encounters but only in 19% of non-screening encounters. BH prescription changes and referrals were more likely after screening (7% and 10%) compared with non-screening encounters (2% and 1%).

The process of screening – even when negative – seems to increase brief interventions and referrals. Looking at only the patients who had negative screens, screening encounters were associated with a roughly two-fold increased intervention rate compared with non-screening encounters for the same patient: problem list updates (9% v 6%); use of a BH encounter diagnosis (31% v 19%); BH medication adjustment (4% v 2%); and referrals for BH treatment (7% compared with 1%) (►[Table 4]).

Compared to negative screens, positive screens led to 2–5 times higher rates of clinical intervention: problem list updates (24% v 9%); use of a BH encounter diagnosis (73% v 31%); use of BH medication (20% v 4%), and new BH referral (22% v 7%).

5. Discussion

We developed and implemented a behavioral screening system using a CDS tool for the collection and reporting of behavioral health data across four primary care practices. There were no clinically significant age or gender differences between screened patients and comparison patients (►[Table 3]). There may have been a selection bias in choosing which patients to screen, as would be expected from the non-randomized sampling.

Prior to implementation, nursing staff expressed concern that patients might frequently screen positive in 2 or 3 domains, triggering up to 31 questions and response entry. These concerns were not realized: none of the patients screened positive in all 3 domains, and only 11% screened positive in 2 domains (►[Figure 5]). The overall positive screen rate (positive in any 1 domain) of 21% and the predominance of positive alcohol screens were comparable to previous reports [[21]–[25]].

Both positive and negative screens were associated with increased provider actions. The process of screening appeared to be associated with provider action even with negative screens. These results suggest the possibility that the patient experience may not adequately be captured with the workflows and tools deployed (tests lack sensitivity). Alternatively, the screening may trigger the patient and provider to engage in a richer conversation not fully captured in the screening tools, emphasizing that while technology is important, it does not necessarily mitigate the importance of a close patient-provider relationship.

5.1 Limitations

The greatest limitation with this study is the potential for screening bias. This is a convenience sample of behavioral health screens, collected by motivated providers, and we cannot suggest the sample is representative. The matching of intervention encounters and control patients could have been more rigorous. We assumed that a random sample of 850 patients matched on age range and gender, from within the same clinic, would produce a cohort with similar comorbidities and social determinants of health, but we did not test that assumption. The non-randomized, underpowered study design also prevented us from applying the rigorous statistical analysis preferred for evaluations.

The measures used target a limited set of patient problems requiring behavioral risk or assistance. Collection of these data is just now systematically being implemented in all target clinics. Decisions on this data set were post hoc, not as result of data rules. We do not know the consequence of all positives, or how to determine if a prescription re-order is a dose increase if the provider has used the free-text SIG field (“take 1 tab daily” to “take 2 tabs daily”) rather than discrete fields. Anecdotally, psychiatric medications are more frequently adjusted using the free-text field due to more nuanced dosing instructions. We also cannot comment on the outcomes of physician actions associated with the screening. Specifically, our definition of Brief Intervention (medication adjustment, problem list comments, or encounter diagnosis) is convenient as a computable phenotype, but clearly does not meet clinical definition for an intervention. Manual chart audits or audio recordings of encounters, possibly combined with natural language processing, would result in better analysis if the encounters could be accomplished with manual auditing or natural language processing of the encounter notes or with audio recordings of encounters. This level of analysis was beyond the scope of our project. Similarly, we were unable to assess the outcomes of referrals since these encounters were almost all conducted outside of the clinical system, in private practices, and not on EHRs.

All data were collected in a single Epic EHR environment, and it may be difficult to reproduce the methods in non-Epic EHR systems. In addition, we did not investigate the impact of screening in a face-to-face encounter with nursing staff rather than on a paper or electronic interface.

5.2 Future efforts

We are updating the content of the EHR tools in order to leverage more widely-adopted, validated questionnaires. We will change the alcohol screening from the AUDIT-1 to the AUDIT-3 and the substance abuse screening from the Prescription Misuse Question to a validated screening tool, possibly one that combines questions about alcohol, drug, and prescription use [[27], [28]].

Future efforts need to focus on more systematic sampling strategies across larger sets of patient reported data, including randomized sampling as well as universal screening. We also will work on minimizing intrusion in practice flow by targeting the set of measures that most represents behavioral risk. Part of the informatics assessment will include user experience metrics. We also plan to collect patient reported outcomes outside the encounter using patient portals, interactive voice response, and tablets in the waiting areas; this evaluation will include patient user experience measures as well.

We will need systematically engaged and mature data sites to assess generalizability. We have piloted internet collected behavioral risk data beyond anxiety, depression and substance use. Future work will allow us richer interpretation of the relationship of positives to health status and medical issues.

We are currently programming the screening tool into our Epic EHR to be delivered automatically, sending patients a message to complete annual screening online. Patients with an Epic portal will receive an Epic questionnaire with the screening questions. Other patients will be mailed a letter using Epic batch letter procedure, with the letter containing a link to a website with the screening questions. Results from the online screening will flow back into the EHR via a standard results interface. We are targeting screening of all adult patients 1 week prior to scheduled annual wellness exam.

In the future, for patients who do not access and respond to the measure before the visit, the screening tool will be available on a tablet in the waiting room or exam room at the time of the encounter, also triggered by CDS rather than staff recall. There is capacity for tablet responses to populate the EHR in real time and file to the same flowsheets as our prior work, allowing us to use the same CDS tools. Epic functionality now allows us to have the completed flowsheets activate a silent BPA which in turn updates the HM topic, so we can eliminate using the encounter diagnosis codes for completing the annual HM screening.

6. Conclusions

This study successfully implemented an EHR-based CDS tool for behavioral health screening using a systematic process tested in multiple practices. We demonstrated that implementation in one commercial EHR can support this process, and anecdotally, similar functionality may be possible in other current EHRs. Qualitatively, we demonstrated that the embedded screening tool is functional and accepted by physicians, patients and used by clinic staff. The workflow makes use of nursing staff and automated scoring and recommendations, leaving the PC provider to focus on the treatment plan.

Results suggest that use of these screening tools affects provider behavior and patient care when positive and, to a lesser degree, even when negative. Further data need to be collected from larger more representative samples to assess consistency of findings. We need to explore other delivery methods (e.g., patient portal, web based data collection, or tablets). Patients need to be more involved in determining the use-case design, and CDS algorithms. Addressing and resolving these issues will allow us to understand better how our methodologies might benefit patient outcomes. In addition, further clarification is necessary to understand the observation that negative screens were still associated with physician action.

• References

• 1 The Working Party Group on Integrated Behavioral: Baird M, Blount A, Brungardt S, Dickinson P, Dietrich A, Epperly T, Green L, Henley D, Kessler R, Korsen N, McDaniel S, Miller B, Pugno P, Roberts R, Schirmer J, Seymour D, DeGruy F. Joint principles: integrating behavioral health care into the patient-centered medical home. Ann Fam Med 2014; 12 (02) 183-185.
• 2 Kathol RG. Prevalence of Behavioral Health Providers in Primary Care. Virginia Primary Care Association conference on Integrating Primary Care; 2009; Virginia: 2009
• 3 Kessler R. Identifying and screening for psychological and co morbid medical and psychological disorders in medical settings. Journal of Clinical Psychology 2009; 65 (03) 253-267.
• 4 Kessler R. Effects on mental health treatment initiation of incorporating mental health services into primary care practice. Journal of the American Board of Family Medicine 2011; 25 (02) 255-295.
• 5 Auxier A, Runyan C, Mullin D, Mendenhall T, Young J, Kessler R. Behavioral health referrals and treatment initiation rates in integrated primary care: a Collaborative Care Research Network study. Translational Behavioral Medicine 2012; 2 (03) 337-244.
• 6 Stange KC, Nutting PA, Miller WL, Stange KC, Nutting PA, Miller WL, Jaén CR, Crabtree BF, Flocke SA, Gill JM. Defining and Measuring the Patient-Centered Medical Home. Journal of General Internal Medicine 2010; 25 (06) 601-612.
• 7 Organization AC. Program Analysis. http://wwwcmsgov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/ACO-NarrativeMeasures-Specspdf 2013
• 8 Kessler R, Miller BF, Kelly M, Graham D, Kennedy A, Littenberg B, MacLean CD, van Eeghen C, Scholle SH, Tirodkar M, Morton S, Pace WD. Mental health, substance abuse, and health behavior services in patient-centered medical homes. J Am Board Fam Med 2014; 27 (Suppl. 05) 637-644.
• 9 NCQA. Patient-Centered Medical Home. 2011 [cited 2016 April 14]; Available from: http://www.ncqa.org/tabid/631/default.aspx
• 10 Siu AL; US Preventive Services Task Force (USPSTF), Bibbins-Domingo K, Grossman DC, Baumann LC, Davidson KW, Ebell M, García FA, Gillman M, Herzstein J, Kemper AR, Krist AH, Kurth AE, Owens DK, Phillips WR, Phipps MG, Pignone MP. Screening for Depression in Adults: US Preventive Services Task Force Recommendation Statement. JAMA 2016; 315 (04) 380-387.
• 11 Jonas DE, Garbutt JC, Brown JM, Amick HR, Brownley KA, Council CL, Viera AJ, Wilkins TM, Schwartz MPH CJ, Richmond EM, Yeatts J, Evans TS, Sally D Wood SD, Russell P Harris RP. Screening, Behavioral Counseling, and Referral in Primary Care To Reduce Alcohol Misuse. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012 Jul. (Comparative Effectiveness Reviews, No. 64.) [cited 2016, October 27] Available from: https://www.ncbi.nlm.nih.gov/books/NBK99199
• 12 Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, Nichol K, Woolliscroft J, Henderson WG. Improving residents’ compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA : the journal of the American Medical Association 2000; 284 (11) 1411-1416.
• 13 Kenealy T, Arroll B, Petrie KJ. Patients and computers as reminders to screen for diabetes in family practice. Randomized-controlled trial. J Gen Intern Med 2005; 20 (10) 916-921.
• 14 Peterson KA, Radosevich DM, O’Connor PJ, Nyman JA, Prineas RJ, Smith SA, Arneson TJ, Corbett VA, Weinhandl JC, Lange CJ, Hannan PJ. Improving Diabetes Care in Practice: findings from the TRANSLATE trial. Diabetes Care 2008; 31 (12) 2238-2243.
• 15 Gill JM, Chen YX, Grimes A, Klinkman MS. Using electronic health record-based tools to screen for bipolar disorder in primary care patients with depression. J Am Board Fam Med 2012; 25 (03) 283-290.
• 16 Ash JS, Sittig DF, Dykstra R, Wright A, McMullen C, Richardson J, Middleton B. Identifying best practices for clinical decision support and knowledge management in the field. Studies in health technology and informatics 2010; 160 Pt 2 806-810.
• 17 Wright A, Sittig DF, Ash JS, Sharma S, Pang JE, Middleton B. Clinical decision support capabilities of commercially-available clinical information systems. JAMIA 2009; 16 (05) 637-644.
• 18 Osheroff JA, Piffer E, Teich J, Sittig D, Jenders R. Improving Outcomes with Clinical Decision Support: An Implementer‘s Guide. 2nd Ed. Chicago, IL: Health Information management Systems Society; 2012. p. 323.
• 19 Ash JS, Sittig DF, Guappone KP, Dykstra, Richard H, Richardson J, Wright A, Carpenter J, McMullen C, Shapiro M, Bunce A, Middleton B. Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study. BMC medical informatics and decision making 2012; 12: 6.
• 20 Sartorius N, Ustun TB, Lecrubier Y, Wittchen HU. Depression comorbid with anxiety: results from the WHO study on psychological disorders in primary health care. The British journal of psychiatry Supplement 1996; 30: 38-43.
• 21 Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine 2001; 16 (09) 606-613.
• 22 Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2: validity of a two-item depression screener. Med Care 2003; 41 (11) 1284-1292.
• 23 Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder. Archives of Internal Medicine 2006; 166: 1092-1097.
• 24 Aalto M, Alho H, Halme JT, Seppä K. AUDIT and its abbreviated versions in detecting heavy and binge drinking in a general population survey. Drug Alcohol Depend 2009; 103 (1–2) 25-29.
• 25 Gomez A, Conde A, Santana JM, Jorrin A. Diagnostic usefulness of brief versions of Alcohol 25. Use Disorders Identification Test (AUDIT) for detecting hazardous drinkers in primary care settings. Journal of Studies on Alcohol 2005; 66 (02) 305-308.
• 26 Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. A single-question screening test for drug use in primary care. Arch Intern Med 2010; 170 (13) 1155-1160.
• 27 NIDA Drug Screening Tool [cited 2017 Feb 2]; Available from: https://www.drugabuse.gov/nmassist/?q=qm_json&pageId=questions_1&pageName=QuickScreen&token_id=194003
• 28 CAGE-AID [cited 2017 Feb 2]; Available from: http://www.integration.samhsa.gov/images/res/CAGEAID.pdf