Nothing for Me or About Me, Without Me: Codesign of Clinical Decision Support

Abstract

Background Partnerships among patients, families, caregivers, and clinicians are critical to helping patients lead their best lives given their specific genetics, conditions, circumstances, and the environments in which they live, work, and play. These partnerships extend to the development of health information technology, including clinical decision support (CDS). Design of these technologies, however, often occurs without a profound understanding of the true needs, wants, and concerns of patients and family members. Patient perspective is important not only for patient-facing applications but for provider-facing applications, especially those intended to support shared decision-making.

Objectives Our objective is to describe models for effectively engaging patients and caregivers during CDS development and implementation and to inspire CDS developers to partner with patients and caregivers to improve the potential impact of CDS.

Methods This article serves as a case study of how two patient activists successfully implemented models for engaging patients and caregivers in a federal program designed to increase the uptake of research evidence into clinical practice through CDS. Models included virtual focus groups, social media, agile software development, and attention to privacy and cybersecurity.

Results Impact on the federal program has been substantial and has resulted in improved CDS training materials, new prototype CDS applications, prioritization of new functionality and features, and increased engagement of patient and caregiver communities in ongoing projects. Among these opportunities is a group of developers and patient activists dedicated and committed to exploring strategic and operational opportunities to codesign CDS applications.

Conclusion Codesign and implementation of CDS can occur as a partnership among developers, implementers, patients, cybersecurity and privacy activists, and caregivers. Several approaches are viable, and an iterative process is most promising. Additional work is needed to investigate scalability of the approaches explored by this case study and to identify measures of meaningful inclusion of patients/caregivers in CDS projects.

Keywords

clinical decision support - patient engagement - codesign - patient and caregiver partnerships - implementation - information security - privacy

Background and Significance

The pandemic surrounding infection from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2 and the condition known as coronavirus disease [COVID]) has heightened awareness of the importance of scientific research to guide decision-making for health care, personal safety, and prevention of acute and chronic illnesses. Decision-making around prevention has included a focus on vaccination, but some populations face tremendous uncertainty when deciding whether or when to receive the vaccination. For example, people who are immunocompromised face uncertainty around the response to vaccination and length of protection. In March of 2021, when preliminary results of a large prospective cohort study revealed low antibody response to COVID vaccination among immunocompromised patients, patient activists in the transplant, chronic kidney disease, dialysis, and immunocompromised communities sought advice from the medical community and from each other.[1] Here we loosely define patient activists as those directly and indirectly impacted by a health challenge, who are knowledgeable about those challenges, and who openly speak about their stories locally and nationwide to help others in similar situations. Media and local/national leaders often listen to patient activists. In one city, a local news outlet published a high-level summary of some of the study's data, which frightened many patients in and around the city and left hospital staff overwhelmed and scrambling to respond to calls and emails.[2] Both patients and clinicians could have understood and navigated this uncertainty together with shared decision-making (SDM) if they had been given the right tools. Other than media stories and a few tweets with links to papers, no tools were available to share study data between clinical teams and their patients, to help patients, families or the activist community get proper, timely, information, or to provide decision support about what to do with this information, if anything. The feeling of being panicked with nowhere to turn and no direct line of support causes helplessness to fall over even the most engaged people. Patients and their families also took to social media and were forced to share extremely personal information, risking the privacy and security of such information. In situations like these, health care decision-making can be fraught with uncertainty and is an opportunity for SDM using decision support tools.

Patient-centered clinical decision support (CDS) is an emerging area for tool development.[3] Meeting the CDS “Five Rights” (i.e., the right information to the right people in the right channels and right formats at the right times) in areas of uncertainty requires sensitivity to and awareness of patient desires, values, capabilities, and limitations.[4] Well-designed, patient-responsive CDS, therefore, necessitates a thorough understanding of these patient-centered elements, which can be gained through a partnership with patients throughout the CDS development and implementation process, including its initial design.[5] Partnering with patients and caregivers is even more important for CDS intended to support SDM, particularly as patient activists and the general public become more knowledgeable about privacy and information security.

Advancing the uptake of best evidence into practice through CDS and learning how to best incorporate patient perspectives have been a priority for the Agency for Healthcare Research and Quality (AHRQ) since the agency launched its CDS initiative in 2016.[6] The initiative is a multicomponent program that encompasses public, web-based infrastructure for sharing interoperable CDS, research grants that use CDS to disseminate evidence into practice at scale, and stakeholder engagement. Engaging stakeholders has taken the form of learning collaboratives, such as the Patient-Centered CDS Learning Network (PCCDS LN) and the recently awarded CDS Innovation Collaborative (CDSiC).[7] [8] This article describes how patient activists have positively impacted the AHRQ CDS initiative to advance patient-centered CDS. The patient activists developed models for patient partnership, which have spurred continued innovation and exploration of CDS codesign. The material presented here is based on content originally presented as a panel by the authors at AMIA's 2021 Clinical Informatics Conference. While patient engagement, even coproduction, in CDS may not be novel, such engagement could be more widespread, varied, creative, and range from beginning to end of the project or product. This includes research and artifact governance, artifact content, design, testing, and dissemination. Engagement and coproduction are complex, not for the faint of heart, yet doable.

Objectives

Our objective was to share approaches for patient partnership in the codesign of CDS and to inspire developers to begin exploring relationships with patient and caregiver communities.

Methods

Description of the Team

Potential pathways for meaningfully engaging patients and caregivers within AHRQ's CDS program originated with two patient activists working with AHRQ and AHRQ's contractor. One patient activist is a patient with multiple sclerosis, care partner for several family members' end-of-life journeys, a nurse for over 40 years, an informaticist, a quality improvement leader, a mentor to leaders and advocates, and a weekly blogger and podcaster for more than 7 years at www.health-hats.com/pod. The other patient activist is a stakeholder inclusion consultant and cybersecurity professional, former patient editor at The British Medical Journal, and the current Board President of the American Living Organ Donor Fund. He is also a Patient-Centered Outcomes Research Institute (PCORI) ambassador, a Medicine X ePatient Scholar, a member of the Information Technology Advisory Committee of the United Network for Organ Sharing Board of Directors, and Board Secretary of the Light Collective.

Approaches for Engaging Patient and Caregiver Communities to Inform Clinical Decision Support Codesign

The patient activists advocated approaches that could engage patient/caregiver communities broadly to provide suggestions and directions for codesign. These approaches and how they were made inclusive of patients and caregiver communities are described here and summarized in [Table 1]. Approaches took advantage of existing mechanisms and channels to which the patient activists already had access.

Table 1 Approaches for engaging patient and caregiver communities to inform clinical decision support codesign

Approach	Description	Results
Virtual focus groups	Engaged advocacy teams working on topics such as chronic pain management and reducing opioid use, including a task force of the Patient Family Advisor Network	Enhanced training materials to help clinicians assess patient readiness for shared decision-making and for actions potentially recommended by the CDS
Social media	Multiple tweets and posts on various platforms asking for feedback, suggestions, and follow up discussion	Over 1,000 hits, 30 substantive comments, and 4 advisors joining CDS projects because of outreach through social media
Agile software development	Assigned stories include patient activists that are also reported on during end of cycle meetings	Consistent check that prioritized tasks lead to features and functionality important to patient and caregiver perspective
Consideration of privacy and security	Review of materials (e.g., reports, implementation guides, Web site content, descriptions of technical architecture, and proposed policies) to indicate attention to privacy and cybersecurity issues that may be of concern for consumers (patients).	Better understanding and priority-setting for explaining the importance of and transparency around technical and non-technical approaches for meeting these concerns, even if technical approaches were sound

Abbreviation: CDS, clinical decision support.

Virtual focus groups. One of the approaches was to raise the topic of CDS in the context of a task force of an existing Patient and Family Advisor (PFA) Network.[9] This task force, which had been meeting for 18 months, consisted of eight people from across the country who served as patient/family advisors to inpatient and outpatient health care institutions on issues related to chronic pain. Over the course of 3 months and three sessions, the task force saw wireframes of a patient-facing pain management CDS application and provided feedback about design, scope of material, choices offered, and gaps that current technology overlooked or was unprepared to fill. One such gap assumed that patients and clinicians had a shared understanding of “personal goals” and the data that might fill clinician free-text fields corresponding to patient goals.

Social media. Feedback from the PFA Network focus group emphasized that pain management decisions often depend on patient goals and priorities. While the tool under review asked about goals, the focus group thought that people had variation in understanding of what personal goals meant and how to formulate goals. In response, and to help the clinical teams, the task force collaborated to develop brief instructional adjunct YouTube videos about dealing with chronic pain and about how people might share their pain goals with their clinical teams.[10] Similarly, invitations to provide input on the topic of patient-centered CDS took place through social media accounts that were already established and followed by members of the patient activist community. One tweet pointed to a web page that offered additional information and invited people to respond and comment.

Agile software development. The agile software development (i.e., iterative and adaptive approach to project management) took place in the context of the AHRQ CDS Connect project, which built a web-based, prototype repository for publicly sharing interoperable CDS resources and developed new standards-based CDS as an exemplar content for the repository.[11] The project operated on 2-week cycles and included patient activist participation in task planning for software engineering, testing, and implementation, with decisions documented during each 2-week cycle.

Assessment of privacy and cybersecurity. A patient activist reviewed multiple resources produced by the CDS program to assess attention to privacy and cybersecurity as areas of particular importance to patients and patient activists. The patient activist review was both a technical and methodological review of how privacy and cybersecurity were described in materials such as user guides, descriptive programmatic materials (e.g., project profiles on the internet), and material presented at scientific conferences. The CDS artifacts produced by the CDS Connect project met all local privacy and security requirements at pilot implementation sites.

Results

Each of the approaches for engaging patient and caregiver communities, through the patient activists, resulted in new CDS resources, altered processes within projects, or built new relationships that provided additional patient perspective as delineated in [Table 1]. In the CDS Connect project, the additional interest resulted in the formation of a patient partnership subgroup of its public clinician- and researcher-focused work group dedicated to exploring goals, methods, and measures for partnering for the codesign of CDS. The collective discussion and proposed materials developed in the subgroup were summarized and shared with the larger work group for their feedback. The proposed materials covered three domains identified as being helpful for those in CDS development and implementation who would like to involve patients and caregivers: (1) an overview of patient partnering, (2) lessons learned when partnering with patients and caregivers, and (3) areas to explore to make patient and caregiver partnering more feasible as a standard practice.

The technical review of privacy and cybersecurity did reveal varying levels of understanding on the part of implementers using cybersecurity protocols, at least in how those protocols were described in available materials (e.g., presentation slides). In addition, projects could have further leveraged principles of privacy by design.

At the program level, lessons learned informed the design and creation of the new CDSiC that will continue to advance the concept of patient-centered CDS. The lessons learned centered on information sharing and access, as well as the recognition that patient and caregiver perspectives can be influential at numerous points in the development and implementation of CDS. Panelists and work group members also identified challenges to including patient and caregiver perspectives: (1) patient/caregiver identification across domains that are often siloed (e.g., cost, clinical evidence, and policy), (2) the need for manageable feedback loops to continue including patient/caregiver perspective after the initial implementation of CDS, and (3) availability of trainings, including relationship building, for inclusion of patients and caregivers. With efforts like the new CDSiC, these lessons and challenges will continue to be explored and shared with the stakeholder community.

Discussion

The incorporation of patient and caregiver perspectives into the design and implementation of CDS gives CDS the greatest potential to positively influence health, especially in situations appropriate for SDM. This case study summarizes successful approaches taken by patient activists to work within a federal program that is advancing patient-centered CDS on a national scale.

Patient and caregiver engagement has been pursued by varying degrees by other initiatives, notably by PCORI and the All of Us campaign.[12] [13] The concept of patient-centered CDS, however, is new, and even less is known about specific strategies tailored toward codesign of CDS.[14] There are several themes among the strategies described. One is the preference toward iterative or sustained engagement. The iterative nature of agile software development, for example, made multiple opportunities for engagement possible, and importantly, the results of that engagement are visible on an ongoing basis in terms of artifacts (e.g., software code and clinician-facing material) generated from cycle to cycle. This led to improved training materials and improved user interfaces for clinicians who face often difficult conversations with patients about managing their chronic pain. Importantly, changes to the software design and/or implementation strategies (e.g., the training materials) were documented into decision logs as sourced from patient activist input. One noted aspect for future research is the added component of privacy and cybersecurity in the development of CDS and engagement with patients. Greater attention to principles of privacy by design and participation by community stakeholders who can provide expert input on the privacy and cybersecurity components should be considered.

Iterative, sustained processes for meaningful patient and caregiver participation are important because progress is likely to be incremental or slow. Most health care communities are just learning how to operationalize patient and caregiver engagement, and the concept may be particularly foreign to information technology teams. Typical CDS applications, for example, because they are intended for clinical applications such as electronic health records, are designed for clinical end-users, not directly for patients or caregivers. However, CDS intended for clinical end-users (e.g., Statin Use for the Primary Prevention of CVD in Adults: Patient-Facing CDS Intervention) may benefit the most by participatory codesign. The need for attention to patient values, preferences, and goals may be obvious for applications clearly targeted for patient audiences, but attention to the same values, preferences, and goals may also be extremely important for applications designed for clinical teams.

Another theme is the recognition that all strategies are dependent upon and should continue to build trust between clinicians, patients, and caregivers. As CDS systems continue to grow in number (e.g., number of rules and clinical recommendations on which they are based), complexity (e.g., artificial intelligence), and types of technologies employed (e.g., integration with sensors and other devices), patients and caregivers are more likely to trust those systems if partnership with patients during their design is robust and transparent.[15] Trust was a major factor identified by patients and caregivers in the PCCDS LN.[16]

Ensuring trust, however, requires the ability to measure trust. One of our key limitations was the lack of a published framework to measure meaningful patient and caregiver engagement during CDS design (e.g., recognition of the need for appropriateness to CDS for patients, but not necessarily including patients).[17] Recently, AHRQ launched the CDSiC, which has several patient activists in key roles to advance a research agenda for measuring trust and patient-centeredness of CDS, among other important areas in the field of CDS. If successful, the CDSiC will build upon the approaches described here and will provide a measurement framework to study the impact of participatory, patient-centered design of CDS. Without a framework to measure meaningful patient/caregiver engagement, it will be difficult for researchers and implementers to know whether the degree of partnership with patient and caregiver communities is sufficient on a project or program level basis.[18] A framework would also help health care and CDS development teams engage patient and caregiver networks (e.g., organized groups of advisors) or individual patients and caregivers. Another limitation of our study was the number of patient activists who developed and implemented the approaches taken by the federal program. Additional experts on patient and caregiver perspective may have led to different or additional approaches.

Developing a more profound understanding of each other's expertise, life experiences, and contexts enabled the federal program to more meaningfully partner with patients and caregivers. Further, learning together what worked and what did not as the partnership grew made us better equipped to describe effective patient and caregiver partnerships to others in our respective spheres of influence. For this we are grateful.

Clinical Relevance Statement

Exploring ways to codesign CDS with patients and caregivers has the potential for more impactful and meaningful CDS that is sensitive to patients' and caregivers' needs, values, and preferences.

Multiple Choice Questions

1. How can relationships with patients and caregivers help frontline informaticists and informatics leaders in their day-to-day operations?

   • Build trust and breakdown inequities

   • Support understanding and adoption of evidence-based practice

   • Foster improved workflows with enhanced relevance, findability, speed, and standards

   • All of the above

   Correct Answer: The correct answer is option d. Relationships with patients and caregivers can help frontline informaticists and informatics leaders in their day-to-day operations in myriad ways. Building trust and breaking down inequities creates an opportunity for successful decision-making regarding care needs (Answer A). Specifically, this trust supports understanding and adoption of evidence-based practices, particularly in situations such as during the COVID pandemic where evidence may be limited (Answer B). The trust and consideration of evidence build to foster improved workflows with enhanced relevance, findability, speed, and standards (Answer C). Ultimately, this relationship building has several benefits to frontline stakeholders (Answer D, correct answer).

2. What is an example of a challenge to shared-decision making through health information security such as CDS?

   • Barriers to information security and interoperability

   • Cannot be done without technology

   • No repository of evidence-based guidelines

   • Everyone wants to share decisions

   Correct Answer: The correct answer is option a. SDM is never easy, but its integration into clinical practice becomes even more challenging when technology is introduced. One challenge is that organizations build proprietary systems, datasets, algorithms, etc., and some organizations choose to not share data or connect their systems. This leads to patient accessibility issues. In this same line of development, information security becomes a barrier as well. Patients need and want to have their privacy and confidentiality preserved and know that any shared information is secure, available, handled properly. Information integrity is a top priority. Answers b through d are incorrect because SDM can be successful without technology (b), there are repositories of evidence-based guidelines available, and SDM requires a personal approach where one size may not fit all (d).

Conflict of Interest

L.F. and E.A.L. have nothing to disclose. Author D.vL. discloses the following: Abridge.com sponsors his podcast, “Health Hats, the Podcast”; consulting relationships with the MITRE Corp. and RTI International; membership in the Board of Governors of the Patient-Centered Outcomes Research Institute. M.M. has had business relationships with Pfizer, CSL Behring, Lyfebulb, Takeda, and Jefferson Health.

Acknowledgments

The views expressed are those of the authors and do not necessarily reflect the official positions of Agency for Healthcare Research and Quality (AHRQ), the U.S. Department of Health and Human Services, or the Patient-Centered Outcomes Research Institute. The authors would like to thank Dr. Laura Marcial of RTI International and Libby Hoy of PFCCpartners.

Protection of Human and Animal Subjects

This article references individual research projects as described on https://cds.ahrq.gov. Each research project was performed in compliance with the U.S. Code of Federal Regulations 45 CFR 46 and reviewed by one or more Institutional Review Boards at the respective institution(s).

• References

• 1 Boyarsky BJ, Werbel WA, Avery RK. et al. Immunogenicity of a single dose of SARS-CoV-2 Messenger RNA vaccine in solid organ transplant recipients. JAMA 2021; 325 (17) 1784-1786
• 2 McCullough M. COVID-19 vaccines don't protect most organ-transplant patients, study finds. The Philadelphia Inquirer. March 17, 2021. Accessed November 8, 2021 at: https://www.inquirer.com/health/coronavirus/covid-19-vaccination-may-be-ineffective-for-immunosupressed-patients-20210317.html
• 3 Agency for Healthcare Research and Quality. Patient-Centered Clinical Decision Support Learning Network. Patient-centered and patient-facing clinical decision support. Accessed November 2, 2021 at: https://pccds-ln.org/node/276
• 4 Campbell R. The five rights of clinical decision support: CDS tools helpful for meeting meaningful use. J AHIMA 2016; 84 (10) 42-47
• 5 Vaughn LM, Jacquez F. Participatory research methods—choice points in the research process. J Particip Res Meth 2020; 1 (01) DOI: 10.35844/001c.13244.
• 6 Lomotan EA, Meadows G, Michaels M, Michel JJ, Miller K. To Share is Human! Advancing evidence into practice through a national repository of interoperable clinical decision support. Appl Clin Inform 2020; 11 (01) 112-121
• 7 Agency for Healthcare Research and Quality. Patient-Centered Clinical Decision Support Learning Network. Accessed June 6, 2022 at: https://digital.ahrq.gov/ahrq-funded-projects/patient-centered-outcomes-research-clinical-decision-support-learning-network
• 8 Agency for Healthcare Research and Quality. Clinical Decision Support Innovation Collaborative. Accessed June 6, 2022 at: https://digital.ahrq.gov/ahrq-funded-projects/clinical-decision-support-innovation-collaborative-cdsic
• 9 The Patient Family Advisor (PFA) Network. The PFANetwork: improving healthcare together. Accessed November 8, 2021 at: https://pfanetwork.org
• 10 Van Leeuwen D. Managing Chronic Pain: Personal Goals, Pain Goals. Accessed November 8, 2021 at: https://youtu.be/PGWQU_tYXfc
• 11 Agency for Healthcare Research and Quality. Patient-Centered Outcomes Research Clinical Decision Support: Repository. Accessed November 2, 2021 at: https://cds.ahrq.gov/cdsconnect/repository
• 12 Patient-Centered Outcomes Research Institute. Public and Patient Engagement. Accessed November 8, 2021 at: https://www.pcori.org/about-us/our-programs/engagement/public-and-patient-engagement
• 13 Klann JG, Joss MAH, Embree K, Murphy SN. Data model harmonization for the All Of Us Research Program: transforming i2b2 data into the OMOP common data model. PLoS One 2019; 14 (02) e0212463
• 14 Grindell C, Tod A, Bec R. et al. Using creative co-design to develop a decision support tool for people with malignant pleural effusion. BMC Med Inform Decis Mak 2020; 20 (01) 179
• 15 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
• 16 Agency for Healthcare Research and Quality. Patient-Centered Clinical Decision Support Learning Network. Trusted Use Frameworks Working Group (TFWG). Accessed November 2, 2021 at: https://pccds-ln.org/tfwg
• 17 Ash JS, Chase D, Baron S. et al. Clinical decision support for worker health: a five-site qualitative needs assessment in primary care settings. Appl Clin Inform 2020; 11 (04) 635-643
• 18 Orenstein EW, Muthu N, Weitkamp AO. et al. Towards a maturity model for clinical decision support operations. Appl Clin Inform 2019; 10 (05) 810-819

Publication History

Received: 14 December 2021

Accepted: 17 April 2022

Article published online:
29 June 2022