A Tool to Identify and Engage Patients on Risky Opioid Regimens

Keywords

opioid analgesics - chronic pain - veterans - medical informatics

Background and Significance

Over the past two decades, opioids have been linked to nearly 650,000 overdose fatalities in the United States,[1] with a record high number of predicted deaths in 2022.[2] Opioids have been one of the most common treatments for chronic noncancer pain. However, long-term opioid therapy, higher opioid doses, and concurrent opioid–benzodiazepine prescribing all increase risk for addiction and overdose.[3] [4] [5] To reduce opioid-related harms, in 2016 the Centers for Disease Control and Prevention (CDC) revised recommendations for patients living with chronic pain on opioid therapy.[6] These guidelines caution against prescribing opioids greater than 90 mg morphine equivalent daily doses (MEDD) or prescribing concurrent opioid and benzodiazepine medications. The updated 2022 CDC guideline eliminated mention of a specific threshold but still emphasized caution regarding high-dose therapy.[7] Use of nonpharmacological therapies (NPTs), such as cognitive behavioral therapy or physical therapy, are also encouraged.[6] [7] A growing body of evidence suggests that NPTs plus gradual dosage reduction for high-risk opioid regimens have been associated with improved pain control and functioning.[8] [9] [10] [11] [12] [13] [14]

As the largest integrated health system in the United States, the Department of Veterans Affairs (VA) has pursued several innovative strategies to address risky opioid prescribing regimens.[15] [16] Veterans have been disproportionately impacted by the opioid epidemic, as approximately 50% experience chronic pain[17] compared with 20% of U.S. adults.[18] Notably, the VA's Opioid Safety Initiative, begun in 2013, has demonstrated reduced use of opioids and improved safe prescribing through data-driven education, pain management, risk mitigation, and addiction treatment strategies.[19] As part of this initiative, a dashboard tool was developed to provide prescriber- and patient-level data on risky opioid and concomitant opioid–benzodiazepine regimens.[15] Over a 2-year period, use of the dashboard was associated with a 16% reduction in patients on high-risk regimens.[15] Additionally, the Stratification Tool for Opioid Risk Mitigation (STORM) tool estimates risk of overdose and suicide and provides tailored, evidence-based mitigation strategies.[20] STORM has demonstrated value in identifying high-risk patients and was associated with decreased all-cause mortality.[21]

However, a key challenge has been engaging patients and facilitating shared decision-making on pain management options with care teams, which was also emphasized in the updated 2022 CDC guideline.[7] Direct-to-patient outreach ahead of upcoming appointments has shown promise in engaging patients in discussions around medication safety and reducing risky regimens.[22] Given the time constraints faced by medical teams, innovative tools are needed to reduce the burden on clinicians while facilitating patient empowerment in safe and effective chronic pain management. Examining user acceptance and usefulness during the design and implementation of these tools is critical to reduce clinician burden and enhance usability, a major hurdle of health information technology adoption.[23]

Objectives

In this clinical informatics case report, we describe the development and implementation of a web-based case-finding tool to identify and outreach to patients on risky opioid regimens who might benefit from timely engagement in a patient-centered assessment of risks and benefits of their current opioid regimen. We examined user acceptance of the tool to inform optimizations for future implementations.

Methods

Primary Care-Integrated Pain Support Initiative

A primary care-integrated pain support (PIPS) initiative was developed to mitigate risk from certain risky opioid regimens and promote engagement in NPTs among patients with chronic pain.[24] [25] This pharmacist-led approach was based on a collaborative care model to improve coordination of pain care between primary care physicians, pharmacists, and clinical staff.[15] PIPS was targeted towards patients who: (1) were receiving opioid prescriptions of ≥90 mg MEDDs or combination opioid–benzodiazepine prescriptions[6] and (2) had an upcoming routine primary care appointment within 2 to 3 weeks. Prior to the appointment, clinic staff mailed eligible patients a letter that described safety concerns with high-risk opioid prescriptions and promoted awareness of NPTs (see [Fig. 1] for the full letter). Patients were encouraged to bring the letter to the appointment, which is in line with a prior approach to facilitate patient–provider conversations on risky medication regimens.[22] During the appointment, providers could review medication safety and refer patients to the PIPS intervention. If referred, a pharmacist met with the patient to discuss NPT options and a dose reduction plan if mutually agreed upon. As part of this multifaceted initiative, we designed and developed a case-finding tool to facilitate automated identification and outreach to patients. This study followed guidance for writing clinical informatics case reports.[26]

Fig. 1 Patient outreach letter template.Note: All figures do not contain any identifiable patient information (it is imaginary).

Design and Development of Case-Finding Tool

Pain subject matter experts (W.B., A.M.M.) identified three core functions for the tool: (1) identification of patients prescribed high-risk medication regimens with an upcoming primary care appointment, (2) generation of letters to mail to eligible patients, and (3) maintaining a record of patients who had been contacted. These components were iteratively built and discussed among team members for the initial version of the tool. Data elements were also examined to aid in query development and dashboard display ([Table 1]). As the tool was intended to support implementation of the PIPS initiative across three primary care clinics, it did not integrate with the electronic health record (EHR), which would require substantial financial and computing resources to build and test across sites.

Table 1 Data elements used in case-finding tool

Domain	Element	Description
Patient	ID	Unique patient identifier
	Name	First and last name
	Address	Most recent address
	Phone number	Most recent phone number
Diagnosis	Diagnosis code	ICD-9 and ICD-10 codes for chronic pain
	Date	Date of diagnosis
Medication	ID	Unique medication identifier
	Name	Medication name
	Class	Medication class (i.e., opioids, benzodiazepines)
	Dosage	Medication dosage
	Start date	Medication start date
	End date	Medication end date
	Days supply	Number of days supply
	Status	Medication status (i.e., active, suspended with medication end date less than current date, expired with medication end date later than current date)
	Morphine equivalent daily dose	Calculation based on morphine-based medications prescribed over one year period
Appointment	ID	Unique appointment identifier
	Type	Stopcodes for primary care/medicine or general internal medicine
	Date	Upcoming appointment within <3 wk of current date
	Medical facility	Health care system facility code
	Clinic	Name of clinic
Provider	ID	Unique provider identifier
	Name	Name of provider
	Type	Primary care provider
Letter tracking	Mailed flag	Yes/no flag to indicate letter mailing (inputted by clinic staff)
	Date mailed	Date letter was mailed (inputted by clinic staff)
Consult Tracking	Consult flag	Yes/no flag to indicate consult with pharmacist (inputted by clinic staff)
	Date of consult	Date of initial consult (inputted by clinic staff)

We developed an extract, transform, and load process using the VA's Corporate Data Warehouse (CDW) to extract eligible patients and relevant data elements daily. These data were used to populate the dashboard, which used Microsoft SQL Server Reporting Services web-based framework ([Fig. 2]). The dashboard was hosted on a VA internal website that clinic staff accessed with a user name and password. Site-specific letter templates could be autopopulated with patient information, including name and address, and sent to a printer. The dashboard included write back functionality where clinic staff could input if the letter was mailed. [Fig. 3] illustrates a sample patient-level view of the dashboard.

Fig. 2 System architecture. EHR, electronic health record.

Fig. 3 Sample patient-level dashboard view.

Implementation

The case-finding tool was implemented in primary care clinics across three VA health care systems: two in the South and one in the West. It was deployed between March 1, 2017 and March 1, 2019 in one Southern and one Western site and during May 1, 2018 to May 1, 2020 in one Southern site. Initial training was provided to end users (i.e., pharmacist, pharmacy technician, research assistant) on accessing the dashboard, filtering patients, and printing letters. Users accessed the patient list at least once every 2 weeks to review patients with upcoming appointments. The developers provided ongoing support throughout the implementation period for each site. [Fig. 4] describes the patient identification, engagement, and tracking process.

Fig. 4 Patient identification, outreach, and tracking process. MEDD, morphine equivalent daily doses.

Evaluation

Qualitative feedback from users (pharmacist, pharmacy technician, research assistant) was gathered through interviews and user testing to examine perceived ease of use, usefulness, and refinements for the tool. This process was guided by the Technology Acceptance Model, which has been applied to numerous studies to explain acceptance and adoption of health information technologies.[27] Perceived ease of use relates to the degree to which the person believes it is easy (mental and physical effort) to use the system, whereas perceived useful is the extent to which the person thinks that their performance will be increased by using the system.[27] One interviewer elicited feedback from users during the implementation period, and notes were corroborated to identify input related to perceived ease of use and usefulness. Data from the CDW were used to provide context on the demographic and clinical characteristics of patients who were mailed the letter.

Results

Over a 24-month period, the case-finding tool identified and mailed 1,315 letters among 1,125 patients ([Table 2]). Letters were mailed multiple times as patients became reeligible for the PIPS intervention with medication changes. Half of the patients were 45 to 64 years, and the majority were male (89%) or White non-Hispanic (78%). Approximately 42% met the 90 mg MEDD opioid criteria, 58% met the benzodiazepine coprescription criteria, and 5% met both criteria. Approximately 59% of patients were from southern health care sites and 41% from the western health care site.

Table 2 Characteristics of letter recipients (n = 1,125)

	n (%)
Age
18–44	96 (9%)
45–64	567 (50%)
65+	462 (41%)
Gender
Women	128 (11%)
Men	997 (89%)
Race and ethnicity
Black, non-Hispanic	120 (11%)
White, non-Hispanic	883 (78%)
Hispanic	45 (4%)
Other	77 (7%)
Medication[a]
Opioid ≥ 90 mg MEDD	473 (42%)
Coprescribed opioid and benzodiazepine	656 (58%)
Both criteria above	60 (5%)
Site
First Southern Healthcare System	491 (44%)
Last Southern Healthcare System	170 (15%)
Western Healthcare System	464 (41%)

Abbreviation: MEDD, morphine equivalent daily doses.

^a Medication categories are not mutually exclusive (i.e., 60 individuals received opioid prescriptions of ≥90 mg MEDDs or combination opioid–benzodiazepine prescriptions)

Based on the qualitative stakeholder feedback, the dashboard was easy to use and helpful to identify patients without conducting time-consuming patient chart reviews ([Table 3]). Sorting and filtering functionalities made it easy for clinical staff to identify patients with the riskiest opioid regimens and those who had not yet been sent a letter. It also required minimal routine maintenance and troubleshooting to maintain functionality. The dashboard was able to serve a further purpose as an audit and feedback tool, which is an implementation strategy that summarizes clinical performance data for monitoring, evaluating, and modifying provider behavior.[28] For instance, PIPS project staff were able to, in real time, compare referrals to the intervention with patients identified as eligible and discuss discrepancies between these patient groups with pharmacists and clinicians during monthly site meetings. Although this was an unexpected application of the dashboard, it allowed project personnel to monitor all avenues of patient engagement, such as provider referral to the intervention.

Table 3 Stakeholder feedback on case-finding tool

Domain	Feedback
Perceived ease of use
Sorting and filtering	• Easy to sort patients by daily MEDDs to identify those with highest dosages • Ability to filter patients that had already been mailed letters to quickly identify letters needed
Technical assistance	• Minimal training needed to instruct end users how to access dashboard and use the features (~1 h per clinic) • Limited technical support and trouble-shooting needed over the implementation period (~5 h per clinic)
Perceived usefulness
Patient identification	• Little time required to identify high-risk patients for outreach
Audit and feedback	• Project staff able to examine which patients were being referred to pharmacists
Letter generation	• Little effort required to generate letters from prepopulated template • Minimal staffing hours needed to print and mail letters (<10 min/wk)
Patient engagement	• Difficult to track which patient received a letter and brought it to appointment

Abbreviation: MEDD, morphine equivalent daily doses.

Stakeholder feedback varied across sites and was iteratively incorporated into the dashboard. For example, the dashboard's initial version did not contain data elements for the date and time of upcoming primary care appointments. Clinicians emphasized the importance of this information to facilitate timely sending of the letter. Also, generation of a separate view for patients who had been sent a letter was requested to improve patient tracking. This tailoring-focused coding and testing took an added average of 50 hours. Additional data elements and system functionalities were iterated on to enrich the dashboard capabilities and user experience.

Discussion

Main Findings

As health care systems continue to focus attention toward patient-centered, precision medicine,[29] leveraging system-wide data can assist in prioritizing high-risk patients to receive evidence-based practices and collaborative care interventions. To our knowledge, this paper is the first attempt to describe the application of a case-finding tool to prioritize, engage with, and track a high-risk patient population. It is crucial to ensure that patients who would most benefit from patient-centered reassessment of risky opioid therapy are promptly provided with these options. Although the dashboard was only one tool within an overall implementation facilitation strategy,[24] it was useful to provide real-time patient eligibility and enrollment numbers to maintain effective prioritization of high-risk patients. Without a dashboard, it would be unrealistic to accomplish the identification and engagement tasks without a large project or clinical staff dedicated to manual patient record review.[30] [31] This work highlights the value of iterative design and development that can incorporate changing user needs that vary across clinic staff and sites.[32] [33]

We identified several areas for future system enhancements based on the stakeholder feedback and implementation experience. It was difficult for clinic staff to determine if a letter was received prior to the patient's appointment, which prevented measurement of the extent to which this outreach effort reached patients. Future improvements should incorporate additional tracking workflows and functionality, such as asking patients during the visit if they received the letter and incorporating electronic outreach modalities with read receipts. In addition, dashboard enhancements could incorporate visual affordances such as color-coding, icons, and other visual displays to highlight changes in medications or eligibility status. Prior studies have found that visualization techniques can improve usability and enhance decision-making.[34] [35] [36] [37] Furthermore, inclusion of additional patient-centered factors, such as preferences, goals, or other patient-generated health data, may be valuable to involve patients in shared decision-making.[38] The VA Lighthouse Application Programming Interfaces (APIs) contain tools to build interoperable digital solutions that can help patients and their clinicians manage their health.[39] Supporting the widely used Fast Healthcare Interoperability Resources (FHIR) data standard,[40] the Lighthouse APIs could make it possible to enhance existing clinician-facing tools through incorporation of patient-contributed data, integration with the EHR, and use of “hooks” to alert clinic staff to changes. Notably, as the VA transitions to the Oracle Cerner EHR, use of standards-based FHIR APIs is critical to support clinical tools that can work across different EHRs. Future work could leverage interoperable dashboards to expand identification and outreach efforts to all patients on risky opioid regimens.

Limitations

This study had a few limitations to acknowledge. First, we lacked detailed information on the patients who received the letter or scheduled an appointment upon letter receipt, as this was not tracked consistently across the three sites. Enhancing workflow tracking and functionalities in future versions of the tool will be valuable to understand the reach and effectiveness of the intervention. Second, we focused on identification of patients who were prescribed opioids greater than 90 MEDD, which is concordant with the 2016 CDC recommendations for patients living with chronic pain on opioid therapy.[6] However, there are a number of additional clinical and social factors that may contribute to risky opioid regimens that should be considered to optimize the tool and patient outreach efforts. Third, at the time of this study, we lacked data from Prescription Drug Monitoring Programs (PDMP), which are state-level databases that allow health providers to track controlled substance prescriptions. There is still not a nationally standardized PDMP, as they are regulated by each state with varying requirements for incorporating PDMP data into EHRs. Obtaining PDMP data for the three sites in different states at the time of tool development would have been prohibitively resource intensive and cause delays in implementation. Therefore, MEDDs were calculated from VA-prescribed medications only, and it could be possible that some patients did not receive a letter if their medications were prescribed from another health care system. Future iterations of our tool and MEDD calculations should incorporate PDMP data, as the CDC recommends use of these data to assess cumulative opioid dosages and medication combinations.[7] Fourth, reporting of changes in coprescribing of opioids and benzodiazepines or changes in MEDD were outside of the scope of this paper given the larger implementation trial examining the impact of implementation strategies on uptake of the intervention. These findings will be reported in a separate analysis. Fifth, we assessed acceptance of the tool through qualitative feedback and did not measure other aspects of usability that have also been shown to influence acceptance and use (e.g., efficiency, effectiveness).[41] Lastly, this tool was implemented across VA primary care clinics, and the feedback may not generalize to other clinics or health systems that may be considering implementing similar tools.

Conclusion

Developing and implementing case-finding clinical informatics tools can facilitate patient identification and direct-to-patient outreach. The tool was valuable to prioritize high-risk patients prescribed risky opioid medication regimens across three health care systems. Stakeholders found the tool easy to use, valuable to identify patients and prepopulate their letters, and it required minimal maintenance and troubleshooting over the 24-month implementation period. This work emphasizes the value of iterative user-centered design that can adapt to user needs and be tailored for each clinic. Incorporating additional tracking features and workflows, visual displays, and patient-level measures may be valuable to reduce workload burden and enhance patient-centered care.

Clinical Relevance Statement

Optimizing the tools and information used to deliver care is essential to enhancing patient-centered care and reducing the time burden for busy clinical staff. Our findings have implications for health systems considering clinical informatics interventions that prioritize, engage, and track high-risk patient populations.

Multiple-Choice Questions

1. How was acceptance of the case-finding tool measured?

   • Usability evaluation

   • Card sorting

   • Survey

   • User testing sessions and interviews

   Correct Answer: The correct answer is option d. User testing and interviews were conducted among end users to examine acceptance, specifically perceived ease of use and usefulness, as guided by the Technology Acceptance Model.

2. What was an unexpected application of the case-finding tool?

   • Engaging families and caregivers

   • Serving as an audit and feedback tool

   • Promoting health information exchange

   • Reducing appointment no-shows

   Correct Answer: The correct answer is option b. Audit and feedback tools are a type of implementation strategy that summarize clinical performance data for monitoring and evaluating provider behavior. In this study, the tool allowed project personnel to monitor all avenues of patient engagement, such as provider referral to the intervention.

• References

• 1 Centers for Disease Control and Prevention. Mortality multiple cause files. 2021. Accessed November 10, 2022 at: www.cdc.gov/nchs/data_access/vitalstatsonline.htm#Mortality_Multiple
• 2 Centers for Disease Control and Prevention. Provisional drug overdose death counts. 2023. Accessed May 1, 2023 at: https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm
• 3 Coyle DT, Pratt CY, Ocran-Appiah J, Secora A, Kornegay C, Staffa J. Opioid analgesic dose and the risk of misuse, overdose, and death: a narrative review. Pharmacoepidemiol Drug Saf 2018; 27 (05) 464-472
• 4 Bohnert AS, Logan JE, Ganoczy D, Dowell D. A detailed exploration into the association of prescribed opioid dosage and overdose deaths among patients with chronic pain. Med Care 2016; 54 (05) 435-441
• 5 Volkow ND, McLellan AT. Opioid abuse in chronic pain—misconceptions and mitigation strategies. N Engl J Med 2016; 374 (13) 1253-1263
• 6 Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain—United States, 2016. MMWR Recomm Rep 2016; 65 (01) 1-49
• 7 Dowell D, Ragan KR, Jones CM, Baldwin GT, Chou R. CDC clinical practice guideline for prescribing opioids for pain—United States, 2022. MMWR Recomm Rep 2022; 71 (03) 1-95
• 8 Frank JW, Lovejoy TI, Becker WC. et al. Patient outcomes in dose reduction or discontinuation of long-term opioid therapy: a systematic review. Ann Intern Med 2017; 167 (03) 181-191
• 9 Mackey K, Anderson J, Bourne D, Chen E, Peterson K. Benefits and harms of long-term opioid dose reduction or discontinuation in patients with chronic pain: a rapid review. J Gen Intern Med 2020; 35 (Suppl. 03) 935-944
• 10 Fishbain DA, Pulikal A. Does opioid tapering in chronic pain patients result in improved pain or same pain vs increased pain at taper completion? a structured evidence-based systematic review. Pain Med 2019; 20 (11) 2179-2197
• 11 Huffman KL, Rush TE, Fan Y. et al. Sustained improvements in pain, mood, function and opioid use post interdisciplinary pain rehabilitation in patients weaned from high and low dose chronic opioid therapy. Pain 2017; 158 (07) 1380-1394
• 12 Nicholas MK, Asghari A, Sharpe L. et al. Reducing the use of opioids by patients with chronic pain: an effectiveness study with long-term follow-up. Pain 2020; 161 (03) 509-519
• 13 Murphy JL, Clark ME, Banou E. Opioid cessation and multidimensional outcomes after interdisciplinary chronic pain treatment. Clin J Pain 2013; 29 (02) 109-117
• 14 Sullivan MD, Turner JA, DiLodovico C, D'Appollonio A, Stephens K, Chan YF. Prescription opioid taper support for outpatients with chronic pain: a randomized controlled trial. J Pain 2017; 18 (03) 308-318
• 15 Lin LA, Bohnert ASB, Kerns RD, Clay MA, Ganoczy D, Ilgen MA. Impact of the Opioid Safety Initiative on opioid-related prescribing in veterans. Pain 2017; 158 (05) 833-839
• 16 Wiedemer NL, Harden PS, Arndt IO, Gallagher RM. The opioid renewal clinic: a primary care, managed approach to opioid therapy in chronic pain patients at risk for substance abuse. Pain Med 2007; 8 (07) 573-584
• 17 Kerns RD, Otis J, Rosenberg R, Reid MC. Veterans' reports of pain and associations with ratings of health, health-risk behaviors, affective distress, and use of the healthcare system. J Rehabil Res Dev 2003; 40 (05) 371-379
• 18 Dahlhamer J, Lucas J, Zelaya C. et al. Prevalence of chronic pain and high-impact chronic pain among adults—United States, 2016. MMWR Morb Mortal Wkly Rep 2018; 67 (36) 1001-1006
• 19 Gellad WF, Good CB, Shulkin DJ. Addressing the opioid epidemic in the United States: lessons from the Department of Veterans Affairs. JAMA Intern Med 2017; 177 (05) 611-612
• 20 Oliva EM, Bowe T, Tavakoli S. et al. Development and applications of the Veterans Health Administration's Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safety and prevent overdose and suicide. Psychol Serv 2017; 14 (01) 34-49
• 21 Strombotne KL, Legler A, Minegishi T. et al. Effect of a predictive analytics-targeted program in patients on opioids: a stepped-wedge cluster randomized controlled trial. J Gen Intern Med 2023; 38 (02) 375-381
• 22 Tannenbaum C, Martin P, Tamblyn R, Benedetti A, Ahmed S. Reduction of inappropriate benzodiazepine prescriptions among older adults through direct patient education: the EMPOWER cluster randomized trial. JAMA Intern Med 2014; 174 (06) 890-898
• 23 Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19 (03) 413-422
• 24 Midboe AM, Martino S, Krein SL. et al. Testing implementation facilitation of a primary care-based collaborative care clinical program using a hybrid type III interrupted time series design: a study protocol. Implement Sci 2018; 13 (01) 145
• 25 Becker WC, Mattocks KM, Frank JW. et al. Mixed methods formative evaluation of a collaborative care program to decrease risky opioid prescribing and increase non-pharmacologic approaches to pain management. Addict Behav 2018; 86: 138-145
• 26 Sakumoto M, Nielson JA. Guidance for writing a clinical informatics case report. ACI Open 2022; 6 (01) e44-e48
• 27 Rahimi B, Nadri H, Lotfnezhad Afshar H, Timpka T. A systematic review of the technology acceptance model in health informatics. Appl Clin Inform 2018; 9 (03) 604-634
• 28 Powell BJ, Waltz TJ, Chinman MJ. et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015; 10 (21) 21
• 29 Ginsburg GS, Phillips KA. Precision medicine: from science to value. Health Aff (Millwood) 2018; 37 (05) 694-701
• 30 Govindan M, Van Citters AD, Nelson EC, Kelly-Cummings J, Suresh G. Automated detection of harm in healthcare with information technology: a systematic review. Qual Saf Health Care 2010; 19 (05) e11
• 31 Beauharnais CC, Larkin ME, Zai AH, Boykin EC, Luttrell J, Wexler DJ. Efficacy and cost-effectiveness of an automated screening algorithm in an inpatient clinical trial. Clin Trials 2012; 9 (02) 198-203
• 32 Chandran S, Al-Sa'di A, Ahmad E. Exploring user centered design in healthcare: a literature review. 4th International Symposium on Multidisciplinary Studies and Innovative Technologies. 2020;1–8.
• 33 Chen E, Leos C, Kowitt SD, Moracco KE. Enhancing community-based participatory research through human-centered design strategies. Health Promot Pract 2020; 21 (01) 37-48
• 34 Dowding D, Merrill J, Russell D. Using feedback intervention theory to guide clinical dashboard design. AMIA Annu Symp Proc 2018; 2018: 395-403
• 35 Taieb-Maimon M, Plaisant C, Hettinger AZ. et al. Increasing recognition of wrong-patient errors through improved interface design of a computerized provider order entry system. Int J Hum Comput Interact 2018; 34 (05) 383-398
• 36 Allen KS, Danielson EC, Downs SM. et al. Evaluating a prototype clinical decision support tool for chronic pain treatment in primary care. Appl Clin Inform 2022; 13 (03) 602-611
• 37 Harle CA, DiIulio J, Downs SM. et al. Decision-centered design of patient information visualizations to support chronic pain care. Appl Clin Inform 2019; 10 (04) 719-728
• 38 Dullabh P, Sandberg SF, Heaney-Huls K. et al. Challenges and opportunities for advancing patient-centered clinical decision support: findings from a horizon scan. J Am Med Inform Assoc 2022; 29 (07) 1233-1243
• 39 Department of Veterans Affairs. Lighthouse APIs. 2023. Accessed May 1, 2023 at: https://developer.va.gov/explore/health/docs/clinical_health?version=current
• 40 Health Level Seven. Enabling health interoperability through FHIR. 2023. Accessed May 1, 2023 at: http://fhir.org/
• 41 International Standard Organization. Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: guidance on usability. 9241-11.1998. Accessed Jan 15, 2023 at: https://www.iso.org/standard/16883.html