Keywords clinical decision support - interoperability - computable guidelines - knowledge management
- shared repository
Background and Significance
Background and Significance
Translation of evidence-based practice into usable and effective clinical decision
support (CDS) is time consuming and expensive.[1 ] Clinical practice guidelines and other evidence-based sources need to be carefully
scrutinized and compared with one another, agreed upon by a clinical community, translated
into highly specific and executable logic, and thoughtfully implemented within workflow
and the local instance of an electronic health record (EHR).[2 ]
[3 ] However, healthcare systems continue to develop CDS in isolation. It has been estimated
that the duplicative effort of translating evidence-based practice into CDS for all
recommended care across all healthcare systems in the United States collectively costs
our healthcare system $25 billion.[4 ]
Depending on the audience, CDS as a concept often carries different meanings. CDS
is not simply the end product of a translation or even implementation process. The
Five Rights framework has been effective in communicating the overall purpose of CDS
to a wide spectrum of audiences, including to patients and caregivers.[5 ] By describing CDS as a process that provides the right information to the right
audiences using the right channels and formats at the right times during work flow,
this framework emphasizes that CDS is not actually an app or any one “thing.” Rather,
CDS is a process of many things, weaving together people, technology, and organizations
toward a common goal of improved health and healthcare.
Effectively sharing CDS to eliminate duplicative effort across health care systems,
therefore, requires sharing knowledge about the entire process. Recently, the Patient-Centered
CDS Learning Network (PCCDS-LN) developed an Analytic Framework for Action that emphasizes
learning and puts patients and families at the center of a cycle divided into prioritizing,
authoring, implementing, and measuring CDS.[6 ] Each of these subprocesses is supported by tools and resources—reusable building
blocks of CDS—that are shareable among healthcare systems.
CDS Connect (https://cds.ahrq.gov/cdsconnect ) was conceived, developed, and launched as a publicly available platform to support
such sharing. CDS Connect is part of a larger AHRQ initiative focused on CDS that
began in 2016.[7 ]
Objectives
The goals of the AHRQ CDS initiative are to (1) advance evidence into practice through
CDS and (2) make CDS more shareable, standards based, and publicly available. One
component of the initiative, CDS Connect, functions as a national repository of CDS
and serves as a public resource for healthcare systems, academic researchers, and
informaticists seeking to share and reuse CDS knowledge resources or “artifacts.”
The purpose of this article is to describe the early experience of CDS Connect, including
the content of its repository, standards-based tools for CDS authoring, and use of
the platform by a variety of users. The content presented here is based on a panel
presentation given at AMIA's 2019 Clinical Informatics Conference (CIC) after which
panelists were invited to submit this article. Other components of the AHRQ CDS initiative,
including the PCCDS-LN, grant-funded activity, and evaluation plans, are described
more fully at https://cds.ahrq.gov/ .
Methods
CDS Connect is both a platform as well as a community of contributors and users supported
by open-source tools for authoring, testing, and sharing interoperable CDS ([Fig. 1 ]). AHRQ contracted with the Centers for Medicare and Medicaid Services (CMS) Health
Federally Funded Research and Development Center (Health FFRDC) operated by the MITRE
Corporation to build and maintain CDS Connect. Central to the platform is a repository
of CDS knowledge artifacts, which are discoverable on the website as entries with
descriptive metadata. The term “artifact” refers to tools and resources (e.g., CDS
software, implementation guides, reports on pilot testing, and other accompanying
material) that serve as building blocks when evidence-based practice recommendations
are translated into interoperable CDS. The repository was developed using Drupal,
a free and open source content-management framework with strong support for authoring,
searching, browsing, and engaging community stakeholders. Each artifact is tagged
using the appropriate Medical Subject Headings (MeSH) terms, allowing users to easily
explore artifacts.[8 ] CDS artifacts comprise entries of the repository and represent a range of resources—the
building blocks of the CDS process previously described—that enable healthcare systems
to translate evidence-based practice into CDS regardless of EHR developer or any other
commercially available health information technology (IT) system. The platform leverages
international standards for codifying the CDS wherever possible and disseminates information
about the entire CDS process for adaptation in local healthcare systems.
Fig. 1 Clinical Decision Support (CDS) Connect concept of operations.
Central to this CDS process is the translation of evidence-based practice into interoperable
CDS components, which requires careful consideration and successively more precise
specification. The CDS Connect project followed a translation process that clinical
informaticists often divide into four phases or levels[9 ] ([Fig. 2 ]). Each level corresponds to a potential set of CDS artifacts, for example, a structured
representation of a guideline recommendation in the form of Health Level 7 (HL7) Clinical
Quality Language (CQL) (Level 3). However, how a CDS developer arrived at each level
is equally important. In migrating through the levels of translation and arriving
at progressively more precise specifications, CDS developers—including the CDS Connect
project—make a variety of assumptions, inferences, and other decisions. An account
of this decision-making is critical for trustworthy sharing of CDS where healthcare
systems must decide whether the same assumptions and inferences—both technical and
clinical—are appropriate for their local environments.
Fig. 2 Levels of translation when developing interoperable clinical decision support.[9 ]
To demonstrate use of the platform and to seed it with CDS artifacts for public use,
the project team undertook use cases in each of its first 3 years ([Table 1 ]). Use cases were designed to demonstrate the potential of different types of CDS
technologies that are shareable across systems. For example, the project team developed
a Substitutable Medical Applications, Reusable Technologies (SMART) on Fast Healthcare
Interoperability Resources (FHIR) application for clinicians in year 2 but developed
a CDS Hooks/CQL service that was integrated into a platform for patients in year 3.
The project generated an array of artifacts at the conclusion of each use case demonstration
that comprised an entry on the CDS Connect repository. Artifacts included Level 2
and Level 3 representations of the CDS, highly detailed user implementation guides
that documented decision-making during CDS development, reports on pilot experiences,
downloadable technical files (e.g., CQL), and links to primary sources of evidence
and other external resources.
Table 1
Use cases adopted by the Clinical Decision Support (CDS) Connect project
Project year
1
2
3
Clinical domain(s)
Cardiovascular disease
Chronic pain management
Cardiovascular disease, diabetes, healthful diet, and physical activity
Guideline developer
US Preventive Services Task Force
CDC
US Preventive Services Task Force
Pilot partner
AllianceChicago
OCHIN
b.well Connected Health
EHR
GE Centricity
Epic
Multiple
CDS technology
Custom CQL-based CDS engine
SMART on FHIR
CQL Hooks
Provider environment
Community health center
Community health center
Multiple
Intended CDS target audience
Clinician
Clinician
Patient
Abbreviations: CQL, clinical quality language; EHR, electronic health record; FHIR,
Fast Healthcare Interoperability Resource; GE Centricity, General Electric Centricity;
SMART, Substitutable Medical Applications, Reusable Technologies.
In addition to the repository, the project team developed several pieces of software
designed to help CDS developers and implementers create, test, share, integrate, and
implement evidence-based, interoperable CDS into health IT systems. The CDS Authoring
Tool was designed to help nonsoftware engineers build their own standards-based CDS
logic using the HL7 CQL standard. The tool allows users to construct CDS logic statements,
select and validate value sets and clinical codes through integration with the National
Library of Medicine's Value Set Authority Center, and save artifacts for reuse or
modification in their own user account. While the initial versions of the Authoring
Tool advanced the concept of user-friendly CDS authoring, the tool continues to evolve
to support the needs of the CDS community. Other software includes the CDS Connect
application programming interface, a custom Drupal module that facilitates the upload
of artifacts to the repository, as well as the CQL testing framework, which allows
users to test the CQL they have developed to ensure the logic statements are constructed
properly and return expected results when run against user-created test patients.
[Table 2 ] lists additional software developed through the course of the project. All software
is open source and freely available.[10 ]
Table 2
Open-source software tools developed by the Clinical Decision Support (CDS) Connect
project for authoring, testing, sharing, and implementing interoperable clinical decision
support
CDS Authoring Tool
CQL Services
Application programming interface
Pain Management Summary
CQL Testing Framework
Purpose
A user-friendly web application for creating CDS logic using simple forms and exporting
it as CQL that uses FHIR as its data model
A service framework that supports remote execution of CQL over a custom RESTful API
or a HL7 CDS Hooks API
A Drupal 8 module that provides a RESTful API for uploading draft artifacts to the
CDS Connect repository
A SMART on FHIR app that promotes informed pain management decision-making and can
be integrated with EHRs that support the SMART platform
A testing framework that allows CQL authors to develop and run test cases for validating
CQL-based CDS logic
Health standards used
HL7 CQL, HL7 FHIR, standard terminologies, integration with VSAC
HL7 CQL, HL7 CDS hooks, HL7 FHIR, standard terminologies, integration with VSAC
HL7 FHIR PlanDefinition
SMART on FHIR, HL7 FHIR, standard terminologies
HL7 CQL, HL7 FHIR, standard terminologies, integration with VSAC
Targeted user
Authors who are not familiar with CDS standards but are interested in developing standards-based
CDS logic
Software developers and health IT system integrators
CDS developers
Clinicians
CQL developers
Open source license
Apache 2.0
https://github.com/AHRQ-CDS/AHRQ-CDS-Connect-Authoring-Tool
Apache 2.0
https://github.com/AHRQ-CDS/AHRQ-CDS-Connect-CQL-SERVICES
GPL 2.0
https://github.com/AHRQ-CDS/AHRQ-CDS-Connect-API
Apache 2.0
https://github.com/AHRQ-CDS/AHRQ-CDS-PAIN-MANAGEMENT-SUMMARY
Apache 2.0
https://github.com/AHRQ-CDS/CQL-Testing-Framework
Abbreviations: AHRQ, Agency for Healthcare Research and Quality; API, application
programming interface; CDS, clinical decision support; CQL, clinical quality language;
FHIR, Fast Healthcare Interoperability Resource; SMART, Substitutable Medical Applications,
Reusable Technologies; VSAC, value set authority center.
To support the continued development of the platform and its associated tools, a public
workgroup was established that provides input on features, functionality, prioritization,
and future direction. The workgroup consists of over 140 volunteers representing more
than 80 distinct organizations. Importantly, the workgroup represents multiple stakeholder
perspectives, including EHR developers, CDS developers, patients, clinicians, payers,
federal agencies, academic researchers, and others. The workgroup provides feedback
that drives improvements to the site and sharing process.
Separate from the CDS Connect project, there has been an ongoing effort to quantify
potential efficiencies gained from using shared, interoperable CDS. To effectively
evaluate the contributions of shareable CDS resources, multiple factors need to be
considered. Under contract with AHRQ, MedStar Health (MedStar) has been studying the
difference between developing CDS using a healthcare system's own resources and tools
versus developing CDS supplemented with artifacts and tools available on CDS Connect.
The MedStar project has been framed to evaluate the CDS lifecycle in its current state
(referred to as “isolated CDS build”) and future state (referred to as “shareable
CDS resources build”) through the application of artifacts and the Authoring Tool
available on the platform.
Four different healthcare systems participated in this comparison. The approach allowed
for systematic comparisons and a rigorous evaluation to demonstrate within-site and
between-site analyses to test the efficiencies of shareable CDS resources in multiple
healthcare settings. Methods included stakeholder interviews, participatory data collection,
process mapping, and usability testing to better describe and quantify efficiencies.
While research suggests there is an ideal CDS process, there are many complications
due to organizational differences.[11 ] Collectively, these analyses have been used as the strategic evidence necessary
to build a business case that considers key aspects of an organization's clinical
and business strategies.
Results
CDS Connect currently has over 50 entries corresponding to CDS developed and shared
by academic institutions, private for-profit and private nonprofit organizations,
and federal agencies ([Table 3 ]). The CDS artifacts cover a wide spectrum of clinical domains from highly specialized
areas such as ophthalmology and neurosurgery to more generalized areas such as preventive
medicine. L2 and L3 representations are the most common. The platform website has
had over 75,000-page views since December 2017. CDS artifact documents have been downloaded
from the repository over 5,000 times within the same time period. The CDS Authoring
Tool has 175 registered users. Because the platform does not currently support tracking
of artifacts following download (e.g., whether the artifacts have been operationalized
elsewhere), knowledge about use of the artifacts come from individual contributors,
explained further below.
Table 3
Overview of content on clinical decision support Connect repository
Contributors
Number of entries (as of November 2019)
Clinical domains
Types of clinical decision support
Agency for Healthcare Research and Quality
12
Cardiovascular disease, diabetes, pain management, preventive care
Data summary, event-condition-action rule, risk assessment
Centers for Disease Control and Prevention
7
Infectious disease, pain management
Event-condition-action rule, multimodal
Children's Hospital of Philadelphia
2
Preventive care, refugee health
Multimodal, order set
HLN Consulting, LLC
1
Preventive care
Multimodal
NORC at the University of Chicago/Yale University
1
Infectious disease
Calculator
RTI-UNC Evidence-Based Practice Center
2
Substance use
Alert, smart documentation form
University of Pennsylvania Health System
1
Infectious disease
Reference information
Veterans Health Administration
31
Endocrinology, mental health, neurology, neurosurgery, ophthalmology, preventive care,
rheumatology
Event-condition-action rule, multimodal, order set, smart documentation form
Contributor Experience: Case Studies
One nongovernment author uploaded two artifacts to the repository soon after the site
was first launched. These artifacts have been downloaded 174 times. One of these artifacts
was published with the intent to disseminate, while the other was published as an
experimental model with the hope of receiving feedback and identifying collaborators.
Two sites (both academic sites with connections to large hospital systems and strong
ties to refugee communities) have finished implementing the first artifact (The Refugee
Health Module) with another two sites currently in progress. Numerous other sites
are considering implementation and have provided feedback to make the artifact faster
to implement and to match better with generic workflows while allowing for local customizations.
One other site (another academic hospital based system) that began implementation
was unable to complete the process largely due to organizational changes, namely,
merging with another site. As part of the dissemination process, feedback was obtained
from each implementation site, whether successful or unsuccessful. This feedback was
used to modify the CDS artifact and the implementation guidance provided through CDS
Connect. Feedback was felt to be generalizable and included: incorporation of standard
clinical terminologies, development of a preimplementation checklist, and identification
of points for local customization. Full details of this dissemination effort have
been published elsewhere.[12 ]
The second artifact published by this author (The Healthy Weight Care Assistant) relied
heavily on custom code and site-specific functions. It was published as a means to
seek feedback and collaboration. Publishing this artifact on the repository resulted
in two collaborative agreements with organizations and researchers looking to adapt
or develop similar CDS products through commercial and academic pathways.
Another set of CDS authors has found success with disseminating through CDS Connect.
Their artifact has been downloaded 153 times and has been successfully implemented
at one outside site as an L2 CDS artifact. Feedback provided on this artifact led
to a revised version of the CDS that was implemented locally and subsequently uploaded
to the repository. The feedback received from the remote implementing institution
included supporting alternative medications and medication allergies. Recognizing
the potential of the site, this CDS artifact has also been encoded in a more mature
state (Level 3) using CQL developed through the CDS Authoring Tool.
Evaluations to understand the role of shareable CDS illustrated changes between the
current state and the shareable state of CDS resources. By aggregating the responses
to compare within and between sites, the research team recognized many challenges
in the evaluation of CDS builds—namely, significant differences based on the level
of artifact structure, the type of CDS build, and the institutional priorities. Efficiencies
were noted from stakeholders in the areas of reduced time for tasks, reduced team
sizes, and fewer tasks to be completed as the outputs were already provided in the
publicly available artifacts. However, not all the research sites noted the same gains
in efficiency as indicated by stakeholder discussions and participatory data collection.
Overall reduced time depends on the specific materials available in individual artifacts.
Feedback from the stakeholders regarding overall impressions of the platform and the
build process using the artifacts was positive, with suggestions provided for improved
efficiency. All sites reported that the more executable, structured, and thorough
the information contained in the artifact, the more helpful it was.
Discussion
CDS Connect is a national repository of publicly available, standards-based CDS knowledge
resources designed to foster sharing among CDS developers and implementers. In its
first 3 years, the platform has grown to support four different federal agencies and
multiple external organizations seeking to share and reuse CDS.
Moving toward Shared, Interoperable CDS
The notion of sharing CDS is not new, and previous attempts have been met with varying
success. AHRQ funded two large contracts with a common aim of demonstrating how CDS
can become more systematic and replicable across sites and systems. The Clinical Decision
Support Consortium (CDSC) developed a service-oriented approach to sharing several
hundred CDS rules in a cloud-based system that served eight clinical sites using five
different EHRs.[13 ]
[14 ] The GuideLines Into DEcision Support (GLIDES) project created and demonstrated tools
that were used to translate evidence-based guidelines into CDS in four different healthcare
systems and that were used by four different medical societies during guideline development.[15 ] However, both the CDSC and GLIDES projects identified significant challenges from
social and legal networks that are needed to facilitate sharing among healthcare systems
to gaps in standards that formalize guidelines for implementation as CDS.
One area that has seen growth and adoption is the set of standards to represent CDS,
its underlying logic and its exchange across different IT systems. HL7 FHIR has gained
significant traction as a means of exchanging data to drive CDS systems, including
data models to support that exchange.[16 ] HL7 CQL aims to harmonize logic that supports both quality measurement and CDS and
that has been adopted by CMS in its quality reporting programs.[17 ] HL7 CDS Hooks and SMART on FHIR are increasingly being used to launch CDS as apps
that are external to the host environment.[18 ]
[19 ]
CDS Connect provides a glimpse of the computable guidelines of the future. Another
federal agency, the Centers for Disease Control and Prevention (CDC), has an ongoing
effort, Adapting Clinical Guidelines for the Digital Age, which aims to make it easy
for clinicians to do the right thing by applying guidelines in practice more easily,
quickly, accurately, and consistently.[20 ] Long-lag time, inconsistencies, and inaccuracies in translation from guideline development
through implementation lead to a problem analogous to the “telephone game,” where
multiple translations of guidelines add complexity, opportunity for error, and variation
across sites or providers ([Fig. 3 ]). A multistakeholder group of experts from the entire guideline development and
implementation continuum determined that by redesigning the process to include all
relevant perspectives from the outset, the interoperable and shareable tools may be
developed and disseminated alongside the guidelines. CDS Connect is a critical component
of the strategy to provide reusable, vetted tools for implementers to be able to apply
the scientific evidence in the recommendations more easily, quickly, accurately, and
consistently. This strategy has the potential to improve health outcomes as well as
improve efficiencies in the time and resources used across healthcare organizations
to implement guidelines.
Fig. 3 Redesigning guideline development and implementation.
Sharing CDS through a national repository should be a beneficial experience for both
the developer and the implementer. The developer of content is able to identify collaborators
working on similar content and potentially create partnerships that would enhance
their own CDS. There is also the opportunity to leverage implementer stories and refine
the CDS locally. A national repository provides a publicly accessible and stable location
for developers to use as a reference point for CDS, limiting the time they would need
to spend on personal site maintenance activities. Finally, by preparing a submission
for a national repository, the developer has an opportunity to consider all the lessons
learned during the CDS development process and apply these internally to other projects.
CDS projects of interest are not unique among organizations. Implementers are often
focused on the same or similar health conditions. This overlap has led to duplicative
efforts across multiple sites with a waste of limited talent being asked to perform
the same tasks. Artifacts published on CDS Connect aim to decrease these duplicative
tasks and allow developers to focus on the true unique needs of their health system
and/or innovative CDS. As requests for additional CDS within organizations continue
to increase and development queues grow longer, a limiting factor in getting evidence
into practice via CDS is a skilled workforce. While education and recruitment efforts
are needed, reduction in duplicative tasks may help to leverage available workforce
and hasten the incorporation of evidence-based CDS into EHRs. A vetted and unbiased
repository, supported by standards that advance interoperability, may decrease organizational
resistance to adoption of CDS designed externally. Additionally, as adoption of externally
developed CDS becomes more of a common practice, it is hoped that organizational barriers
to incorporation will also diminish.
Other options for sharing CDS exist, each with their own benefits and limitations
and will continue to evolve. Many CDS developers use vendor repositories. This practice
may be intentional or automatic as organizations are able to auto-populate these repositories
from the clinical environment. These systems allow for direct importation of some
CDS artifacts into an EHR system, but the vendor-supplied CDS often provide limited
guidance, source attribution, search support, pathways for feedback, or lessons learned
from implementation. Cloud-based CDS options also exist and are useful in numerous
settings. However, these may be less customizable for site-specific workflows, and
organizations have been traditionally averse to transmitting personal health information
to cloud-based CDS servers and allowing these to directly trigger actions within the
EHR, which has limited their reach. Some CDS developers have also had success with
commercialization of CDS artifacts, such as order sets or documentation templates.
CDS Connect allows for developers of content to assert ownership rights and therefore
does not negate the rights of commercial CDS developers.
Creating a Business Case for using Shared CDS
The effort to quantify efficiencies using interoperable CDS can help healthcare systems
construct a business case for using shareable CDS. MedStar's evaluation of the CDS
lifecycle from its current state (isolated CDS build) to future state (shareable CDS
resources build) through the application of CDS Connect artifacts demonstrated potential
efficiencies gained in time and resources. The research team created metrics to characterize
and evaluate processes and resources used during the CDS lifecycle. Qualitative metrics
were linked to processes (e.g., testing, iterations) and quantitative metrics were
linked to time (e.g., individual calendar time and team time) and resources (e.g.,
total personnel). The research team developed unique tools to collect data through
stakeholder discussion and surveys for CDS design, development, and deployment. This
resulted in a total of six tools that captured metrics related to the CDS lifecycle.
A primary finding is the CDS builds vary significantly based on site, EHR vendor platform,
and complexity of clinical condition being addressed. Therefore, comparisons between
CDS build both within healthcare systems and between healthcare systems are challenging
and represent an area that deserves further exploration. Results at the time of presentation
at AMIA CIC indicated efficiencies vary based on the level of maturity and will likely
affect its degree of sharing. CDS resources that are accompanied by value sets, implementation
guides, and computable logic are more likely to increase efficiency and thus sharing
between sites. Despite the presence of evidence-based guidelines, all sites spent
a considerable amount of time verifying the credibility of the clinical evidence.
Verification of the evidence occurred for several reasons: (1) participants wanted
confirmation that the evidence originated from sources perceived as trustworthy and
reputable, (2) verification was essential to ensure that the CDS reflected updated
evidence, (3) verification confirmed important information such as patient population
and clinical users, (4) verification confirmed that information matched local site
workflows, and (5) verification ensured that nomenclature for CDS specifications matched
the site-specific EHR and recommendations matched the practices of the local site.
Shareable resources also led to efficiencies during the development stage including
a reduction in the number of iterations and team time spent on iterations. The deployment
stage was not associated with noticeable efficiencies. However, not all the research
sites noted the same gains in efficiency as indicated by stakeholder discussions and
participatory data collection. Published reports indicate time and effort to develop
shareable CDS are substantial but have resulted in improved efficiency during implementation
at subsequent sites.[12 ]
Overall reduced time depends on the specific materials available in individual artifacts.
It is noteworthy that efficiencies only came from the use of mature artifacts with
structured and computer-interpretable code. Mature artifacts with comprehensive clinical
and technical information, recent updates, and code with the reasoning behind it also
produced greater trust. With regards to team makeup, at all sites, the roles were
similar for all CDS builds, with all sites reporting some combination of clinical
team members and technical team members to accomplish the CDS build. Regarding feedback
from the development stakeholders, while team makeup for the shareable build was the
same as their standard team makeup, the scale of the teams was reported to be smaller
at some sites which may reflect the CDS artifact under consideration.
Feedback from the stakeholders regarding overall impressions of CDS Connect and the
build process using the artifacts was positive, with suggestions provided for improved
efficiency. All sites reported that the more executable, structured, and thorough
the information contained in the artifact, the more helpful it would be. Although
some CDS Connect artifacts provide detailed guides for the design and development
phases in the form of evidence reviews and technical codes and wireframes, additional
resources may be useful during deployment. Some of these resources include guides
on technical and user testing, metrics to capture during testing, identifying the
population of end users to target for training, training content and measuring training
effectiveness, and tracking tool usage postdeployment.
Supporting a Trustworthy CDS Ecosystem
The concept of “trust” was identified by stakeholders in reference to the credibility
of the information provided in the artifacts on CDS Connect. The more thorough and
up to date an artifact was, the more credibility the evidence had, and as such, the
more likely it was to be “trusted” by the clinical team members. Similarly, artifacts
with comprehensive specification information and documentation were more likely to
be “trusted” by technical team members.
It is important to acknowledge the importance of trust in a shared repository of interoperable
CDS. There are multiple layers of trustworthy interactions among contributors, users,
developers, implementers, and evaluators of shared, interoperable CDS, as well as
the need to incorporate the perspectives of patients, families, and caregivers. Developed
through a Trusted Use Framework Working Group, the PCCDS-LN recently released “recommendations
for building and maintaining trust in clinical decision support knowledge artifacts.”[21 ] The framework identified 12 key factors for trustworthy interactions, nine areas
of focus or “trust attributes,” and 33 recommendations for each trust attribute. The
CDS Connect project has incorporated many of the recommendations and will continue
to address others.
Conclusion
A new era of shared, interoperable CDS has begun thanks to advances in health IT standards
and publicly available tools and resources. CDS Connect can be a central player in
the dissemination of actionable knowledge, as identified by the CDC-led multistakeholder
initiative on Adapting Clinical Guidelines for the Digital Age. We have highlighted
the experiences of two healthcare systems who are already reaping the benefits of
contributing to a repository of CDS knowledge resources, and we are quantifying efficiencies
gained in CDS development for healthcare systems that reuse and adapt CDS developed
elsewhere.
Clinical Relevance Statement
Clinical Relevance Statement
Clinical informaticists working in healthcare systems, technology companies, academic
settings, and other environments will find a repository of publicly available CDS
and associated open source tools at https://cds.ahrq.gov/cdsconnect . To facilitate trustworthy sharing of CDS knowledge and to build on each other's
experiences, a community of contributors and users of the CDS repository and its tools
exist. All are invited to participate.
Multiple Choice Questions
Multiple Choice Questions
Sharing of CDS between sites has the potential to decrease variability of care between
organizations. By allowing organizations to start CDS development by leveraging shared
source material, publishing shareable CDS addresses has which of the following benefits
(complete the following sentence): because shareable CDS is shareable,
Implementing organizations do not need to consider intellectual property restrictions
since these are handled by the original CDS developer.
It will work the same for most organizations so there is no need to account for differences
in workflow.
People working on similar projects can draw upon each other's work and focus on the
innovative aspects of CDS development.
When you leave your organization you can take it with you and continue to optimal
health care right away.
Correct Answer: The correct answer is option is c. Publishing shareable CDS has many benefits including
being an avenue for dissemination, reduction in waste during development, and potentially
directly linking to executable source code. However, it is not the answer to every
problem with CDS development. Shareable CDS can lead to collaborations and building
upon each other's work would allow for optimization and innovation of the shared material
(answer c, correct answer). CDS developers always need to be aware of intellectual
property considerations, both of the original CDS and of the publishers of the evidence
source (answer a). Shareable CDS may give implementation teams a leg up but do not
address the local workflows or local variable definitions (answer b). Shareable CDS
is shared between organizations and is not tied directly to your personal EHR profile,
so it would not give you the opportunity to take it with you—however fun this would
be (answer d).
An important reason for the limited availability of CDS capabilities is the application-specific
and institution-specific nature of most CDS implementations. Which evaluation leverages
a sociotechnical approach to understand complications due to organizational differences?
Evaluate healthcare system policies and procedures including flowcharts.
Use an ethnographic and economic approach evaluating personnel, resources, workflow,
and usability considerations.
Evaluate the final CDS products to understand technical specifications across CDS
modules.
Use an analytic approach to identify errors in CDS logic and implementation.
Correct Answer: The correct answer is option b. Despite its potential, CDS implementation and actualization
remain nascent due to the many barriers to realizing the full benefits of CDS-facilitated
value improvement. Contributing factors include lack of shareable CDS content, absence
of systematic means to validate content, technical difficulties of sharing CDS across
institutions and EHR systems, and suboptimal user interfaces, implementation choices,
and workflows. An ethnographic and economic approach evaluating personnel, resources,
workflow, and usability considerations provides a well-rounded evaluation that considers
sociotechnical elements of a system. Methods including stakeholder interviews, task
analysis and process mapping, qualitative data collection from CDS designers and developers,
heuristic evaluation, and think-aloud protocol can help identify structure and process
challenges developing CDS in the absence of externally developed resources.