Fast-Tracking Health Data Standards Development and Adoption in Real-World Settings: A Pilot Approach

Abstract

Background Pilot-testing is important in standards development because it facilitates agile navigation of the gap between needs for and use of standards in real-world settings and can reveal the practicalities of implementation. As the implementation and use of health data standards are usually more complicated than anticipated, the Office of the National Coordinator for Health Information Technology (ONC) routinely oversees and organizes relevant pilot projects.

Objectives This article provides an in-depth look into a sample of ONC's standards-focused pilot projects to (1) inform readers of the complexities of developing, implementing, and advancing standards and (2) guide those seeking to evaluate new standards through pilot projects.

Methods The ONC's approach to conducting pilot projects begins with identifying a clinical care need, research requirement, or policy outcome that is not well supported by existing standards through a landscape review. ONC then selects a testing approach based on the identified need and maturity of relevant standards. Next, ONC identifies use cases and sites to pilot-test the relevant standard. Once complete, ONC publishes a report that informs subsequent projects and standards development.

Results Pilot projects presented here are organized into three categories related to their demonstrated focus and related approach: (1) improving standards for presenting and sharing clinical genetic data, (2) accelerating the development and implementation of new standards, and (3) facilitating clinical data reuse. Each project illustrates the pilot approach from inception to next steps, capturing the role of collaboration among standards development organizations, stakeholders, and end-users to ensure standards are practical and fit for purpose.

Conclusion The ONC approach identifies implementation difficulties prior to broader adoption and use of standards, and provides insight into the steps needed to scale use of standards. The ONC's organization of pilot projects serves as a natural accelerator for building communities of practice, often providing a well-connected beneficiary of lessons learned.

Keywords

health information technology - Health Level 7 International - informatics - pilot projects - research - standards - implementation and deployment

Background and Significance

Few health care delivery changes over the past decade in the United States have been as visible as the widespread implementation of health information technology (IT) in hospitals, medical offices, pharmacies, and other settings.[1] [2] Health IT has been seen as a way to improve health care quality, address safety issues, and enable health information exchange needed for coordinated care.[3] [4] [5] [6] [7] [8] [9] [10] Realizing the hopes for improvement from this transformation requires interoperability across health IT systems at different levels, meaning that health information can readily and securely move from place to place and be understood and effectively used across care settings.[11] [12] [13] To make this possible, practical implementation requires the adoption and use of health data standards in combination with supportive policies to help ensure data can be easily understood and efficiently exchanged.[12] [14] [15]

In the absence of industry prioritization and progress on certain standards for health data, it often falls to the government in collaboration with stakeholders to advance the cause of interoperability. The Office of the National Coordinator for Health Information Technology (ONC) is the federal entity responsible for regulation of health IT, including advancing health data standards to enable nationwide health information exchange.[16] As part of its role coordinating health IT policy and advancing health IT use, ONC leads the development of health IT policies and technical strategies that will make research faster, better, and easier to improve patient care and outcomes.[17] [18] [19] [20]

While health data are critical to self-management, clinical care, quality improvement, and public health, they are also very important to scientific research and discovery.[21] In this area, the importance of standards development and implementation is fundamental, as the need for use of standards in health research has been well established.[22] Scientific discovery requires meaningful, semantically comparable data that can be understood, aggregated, and analyzed for correlation by researchers and the software platforms they use. Accelerating and expanding our knowledge can be achieved through improved health data interoperability. Health data standards useful for research include controlled vocabularies to help ensure data are consistently coded and can be consistently interpreted, ensuring semantic interoperability; health data exchange standards to ensure systems can easily locate and share data when appropriate, enabling structural interoperability; and metadata standards that aid in data interpretation and are of particular use in research, helping to achieve both semantic and structural interoperability.[11] [13] [23]

The need for standardization is often accepted in theory, as it is known that investing the resources needed to implement standards has long-term benefit.[8] [24] [25] [26] [27] However, for the most part, standards are not simply instituted across an industry by fiat. In the spirit of walking before running, trial use of standards through pilot-testing can build trust and experience with health data.

The benefits of pilot-testing in health care broadly have been well established. For example, pilot-test results are a key consideration in the assessment of quality indicators in health care and are considered indispensable for measuring practical feasibility, reliability, and validity.[28] [29] For electronic health record (EHR) systems implementation, pilot-testing with end-users is regarded as an important step, as recommended in implementation and evaluation guides such as the ONC Change Package for Improving EHR Usability.[30] [31] During the implementation phase of pragmatic clinical trials, pilot-testing reduces uncertainties.[32] Cognizant of their value, health care agencies across the U.S. Federal government, including the Agency for Healthcare Research and Quality (AHRQ), Centers for Medicare and Medicaid Services, Food and Drug Administration (FDA), National Institutes of Health (NIH), Veterans Health Administration, and ONC, engage in or fund pilot projects.[28] [33] [34] [35] [36] [37] Similarly, pilot-testing standards may help identify implementation challenges that must be addressed prior to implementation and may help advance a given standard's maturity.[38] In this article, we summarize the approach ONC has employed to advance standards development through pilot-testing to serve as a guide for those seeking to evaluate new standards through pilot projects.

A Pilot Approach to Standards Development

The ONC uses a pilot project approach to developing and advancing standards to support research and clinical care, which is illustrated in [Fig. 1]. Precision medicine and patient-centered outcomes research projects in ONC's portfolio have used this pilot project approach to develop and test standards designed to improve the collection, quality, and exchange of health data for clinical care and research.[39] [40] [41] The pilot project approach involves rigorous analysis of the current research ecosystem, its activities, and outputs, and identifies research data gaps or clinical care needs that could be addressed by creating, modifying, improving, or advancing standards. Landscape review and stakeholder engagement establish the need for particular health data standards.

Fig. 1 The Office of the National Coordinator for Health Information Technology pilot approach for standards development and testing.

Once the needs are delineated and validated by stakeholders, the approach to advance relevant standards is selected. Approaches may include the development of implementation guides, testing of existing standards by relevant organizations, and/or harmonization of standards development efforts across stakeholder groups. To ensure that the health data standards are fit for purpose, ONC develops and establishes use cases with stakeholder input in collaboration with potential demonstration sites. Where possible, existing standards and implementation guides are considered as a starting point for improvement or incorporation. Each project defines broad goals to advance standardization of the data of interest to help meet the identified need.

The next step engages demonstration sites to implement and test the relevant standard(s) for selected use cases in sandbox, pre-production, or production environments depending on the need and maturity of the given standard and defined project goals. Participants may also use the opportunity to explore the readiness of a standard in a given environment or for a particular application, including sharing data across organizations. This testing may occur through several activities including the provision of technical assistance from ONC, subject matter experts, or standards development organizations (SDOs), member-supported organizations, and often accredited by the American National Standards Institute (ANSI), who develop and maintain standards to meet industry needs.[42] Demonstration sites may also participate in “Connectathons,” events where stakeholders, including health IT and standards developers, government representatives, and potential users collaboratively develop and test standards implementation. Connectathons are often used by SDOs to rapidly gather feedback on a given standard and can complement similar joint demonstration site development activities conducted in concert with pilot testing.[43] [44] Lastly, the demonstration sites may engage in implementation and testing in and across live health information systems. Participating demonstration sites define testing outcomes specific to their individual use cases.

ONC involves relevant SDOs, established work groups responsible for standards within a domain, experts, stakeholders, and end-users throughout each project as project team members, as consultants, through technical expert panels, or via direct outreach to ensure all important feedback is considered. Engaging these stakeholders is key to any project not only to ensure user needs are being addressed through testing, but also to successfully advance the standard through official SDO approval (i.e., balloting) processes. SDOs have established standards development and maintenance processes typically conducted through work groups.[42] [45] These processes often involve several stages of balloting in which SDO members provide feedback on a draft standard to be incorporated by the work group responsible for the standard and vote whether the draft specification or version or can proceed to publication as a standard.[42] Project timelines need to align with a given SDO's balloting cycle and benefit from the engagement of relevant work group members or from participating at work group-sponsored development activities such as Connectathons.

Once pilot-testing efforts are complete, a thorough and publicly available evaluation report conveys real-world insight back to the key stakeholders, including the relevant SDO. Pilot project outcomes are synthesized from participants' experiences applying or developing the tested standard, and reported progress toward achieving project goals and use case-specific outcomes. The report documents findings relevant to the acceptability, likelihood of adoption, appropriateness, feasibility, resources needed to implement, or factors that might influence sustainability and draw conclusions for the improvement of the standard or its implementation.[46]

These reports also serve as an enduring and findable resource to those seeking to implement similar standards and approaches in the future. The next steps for implementation can include proposed regulations, nonregulatory guidance, further development, or stakeholder engagement with federal partners such as NIH, FDA, and others who have separate authorities relevant to research or whose programs may benefit by adoption and use of tested standards. Additionally, ONC has worked with SDOs such as Health Level Seven International (HL7) to further formally adopt or refer the draft standard. Below, we further illustrate the application and benefits of this approach through pilot-testing focused on advancing standardized sharing of specific data types or reuse of clinical data.

Multiphase Demonstration Project to Advance Standardized Genetic Data Sharing

Clinical genomics is a burgeoning field with an ever-increasing role to play in clinical care and precision medicine. However, absent standards and computable formats, the results of cutting-edge clinical genomic testing cannot be made available at the point of care to patients and their providers or shared for research. Since 2016, ONC has been conducting pilot projects to accelerate the development of the standards and formats needed for clinical genomics through the Sync for Genes project, which serves as an example of a multiphase project evolving alongside the needs of the community.[47] Sync for Genes activities have contributed to the development of the HL7 Fast Healthcare Interoperability Resources (FHIR) standard and provide a foundation for the interoperable exchange of electronic health data, but they require additional building blocks to serve relevant clinical use cases.

The initial standards undertaken in the first phase of pilot testing were the HL7 specifications for clinical genomics, which were recommended for development by the Precision Medicine Task Force and had already been identified as a need by the All of Us Research Program.[48] [49] [50] Five demonstration sites were selected to provide a diverse set of use cases ranging from family health history genetics to patient and bone-marrow donor antigen matching ([Table 1]). At the conclusion of the first phase, the sites had provided clear demonstration of the usefulness of the HL7 FHIR clinical genomic profile and genomic specifications by adapting them to their own individual domains and activities.[51] The results were taken up by the broader standards community as contributions to the newly drafted HL7 FHIR Genomics Reporting Implementation Guide (STU 1) and the Clinical Genomics Domain Analysis Model.[52] [53]

Table 1 The Office of the National Coordinator for Health Information Technology pilot approach to accelerate standardized genetic data sharing

	Project name
Project attributes	Sync for genes phase 1: standardizing genomic data	Sync for genes phase 2: integrating genomic data	Sync for genes phase 3: engaging laboratories
Project goal	Update the genomic specification	Demonstrate connectivity and exchange of genomic data	Test ability to share standardized genomic data generated by laboratories
Standard(s)	HL7 FHIR genomics implementation guide as part of FHIR release 3.0	• HL7 FHIR Clinical Genomics Standard for Trial Use (STU 3) • HL7 V2 messaging standard	HL7 FHIR genomics reporting implementation guide (STU 1)
Approach	Implementation guide testing	Standard testing via demonstration and Connectathon participation	• Implementation guide testing and development • Data format translator tool development
Use case(s)	• Family health history genetics • Sequencing quality and regulatory genomics • Somatic tumor next generation sequencing • Patient and donor matching	• Pharmacogenomics • Patient and donor matching • Newborn screening • Cancer genomic decision support	• Returning results in a clinical research network • Translating human leukocyte antigen report data into FHIR format
Demonstration sites	• Counsyl and Intermountain Healthcare • Food and Drug Administration • Foundation Medicine and Vanderbilt University Medical Center • Illumina • National Marrow Donor Program	• Lehigh Valley Health Network • National Marrow Donor Program • Utah Department of Health • Weill Cornell Medicine	• Baylor College of Medicine Human Genome Sequencing Center • National Marrow Donor Program
Key findings	• Specification is highly flexible and supports multiple use cases • FHIR schemas can be reasonably conformed to specific needs • FHIR resources, documentation, and queries need improvements to better support genetics use cases	• Semantics, community representation, and training are challenges to the standard's development, adoption, and use • Need to increase FHIR expertise in genomic community and genomic expertise in developer community	• The implementation guide is a useful starting point for tailoring customized profiles • FHIR needs to better reflect the complex structure of specialized reports • Many data types are not well supported by the specification
Next steps	• Pilot-testing of additional use cases that integrate multiple demonstration sites or stakeholders in a shared use case • Development and testing of applications that integrate genomic data with clinical information • EHR integration of genomic information from laboratories via FHIR	• Analysis of current legislation and policies and their application to genomic data • Include provenance and device information in the specification	• Enhance specification to support genomic data for additional complex data types • Establish shared meaning across complex genomic variations

Abbreviations: EHR, electronic health record; FHIR, Fast Healthcare Interoperability Resources; HL7, Health Level Seven International; STU, standard for trial use.

Under guidance from subject matter experts and in discussion with federal partners, phase 2 invited a largely new group of demonstration sites to test a heterogeneous set of use cases focused on the use of clinical genomic data at the point of care ([Table 1]). Their activities followed the newly developed HL7 FHIR Genomic Reporting Implementation Guide, which provides direction on how to implement the FHIR Clinical Genomics Specification.[52] To accelerate the identification of gaps in the FHIR Clinical Genomics Specification and identify the types of local modifications needed for implementation, several of the demonstration sites engaged in a 2-day Connectathon. The diverse feedback from these demonstrations proved a valuable contribution to HL7's Clinical Genomic Work Group advancement of the FHIR Clinical Genomics specification from Standard for Trial Use 3 to version 4, for example, which FHIR resource should be used to attach report files. While the demonstration sites were successful in their adaptation of the FHIR Genomic Reporting Implementation Guide for their uses, they highlighted the opportunity for the laboratory systems performing genetic testing to report results as FHIR messages to clinic.[54] As such, phase 3 focused on employing the FHIR Genomics Reporting Implementation Guide to facilitate the exchange of genomic results at the genotype or phenotype level between genomics testing laboratories and health care provider organizations ([Table 1]). The results of this phase provide clear examples of the expanded use of the specification to describe additional data sources to the HL7 Clinical Genomics Work Group.[55] Continuing the expansion of standards used by and for the genomics field, phase 4 will focus on the use of standard FHIR application programming interfaces (APIs) to share genomic information and provide support to patients.

During each year long phase, Sync for Genes activities have provided proof of concept for those seeking to harness genomic data in patient care and for the discovery of new treatments. The multiphase approach has allowed ONC to consistently provide a proving ground, accelerating the generation of feedback for the development of FHIR-based standards for the sharing of genomic data to SDOs such as the HL7 via its Clinical Genomics Work Group. Throughout these activities, the demonstration sites have served as a foundational example to others in the field seeking to implement standards. Each cohort-style phase of the project has given participants a natural networking opportunity as they share insights, challenges, and tactics. Several Sync for Genes participants have joined or even co-chaired the HL7 Clinical Genomics Work Group and have remained active volunteers beyond the conclusion of their pilot projects; illustrating not only the value of pilot projects in developing standards, but also in developing the communities that support them.[56]

Accelerating Standards to Integrate Social and Patient-Generated Health Data into the Electronic Health Record

The ONC has frequently led pilot projects to explore and develop standards for emergent data sources that are typically external to an EHR, where the challenges and needs of a diverse range of stakeholders are diffuse. In these situations, the starting point for ONC is often the initial development of a standard implementation guide or framework followed by immediate pilot testing ([Table 2]).

Table 2 The Office of the National Coordinator for Health Information Technology pilot approach to accelerate social and patient-generated health data standards

	Project name
Project attributes	Advancing standards for precision medicine: mobile health, sensors, and wearables	Advancing SDOH data collection and exchange	Advancing the collection of patient-reported outcomes through health information technology
Project goal	Advance standardized collection and exchange of mobile health, sensor, and wearable data	Advance standardized collection and exchange of SDOH data	Standardize collection, exchange, and integration of PRO data
Standard(s)	HL7 FHIR	IHE ACDC profile	HL7 FHIR
Approach	• Harmonization of standards development • Implementation guide development and testing: Mobile Health Application Data Exchange Assessment Framework, and Functional Requirements for Mobile Health Implementation Guide	Standard development and testing via demonstration and Connectathon	Implementation guide development and testing: patient reported outcomes FHIR implementation guide
Use case(s)	• Remote monitoring of oxygen saturation, blood pressure, and pulse for patients diagnosed with COVID-19 and congestive heart failure • Sharing of body temperature, blood pressure, heart rate, and physical activity to support COVID-19 diagnosis and treatment	Capture of food insecurity, housing, and transportation through assessments and questionnaires	Use of FHIR API to support collection, exchange, and integration of PRO data for CT-enabled PROMIS Physical Function v2.0 questionnaire
Demonstration sites	• Reliant Medical Group • Get Real Health® and athenahealth	Fenway Health in partnership with the University of Washington and athenahealth	• Patient-centered SCAlable National Network for Effectiveness Research • Research Action for Health Network at the Louisiana Public Health Institute
Key findings	• Identified challenges to integrate data collected from patient-facing applications and devices in an EHR • Standards versioning can impact implementation	• IHE ACDC profile can be used to capture and share questionnaire response data to be displayed in an EHR • There is a need for complete and consistent codes and/or terminology	• Standard can support the use of PRO measures • Standards versioning and lack of standardization across measures can impact testing and implementation
Next steps	• Capture additional data elements (e.g., sleep and physical activity) from mobile health, wearables, and sensors • Improve the patient experience when sharing patient-generated health data with providers • Encourage consumer device manufacturers to adopt open standards	Incorporate additional high-impact data classes (e.g., sexual orientation and gender identity, and occupational history) into future standard advancement efforts	• Build metadata for PRO measures to improve interoperability of PRO data • Adapt the implementation guide for other use cases • Expand the implementation guide to accommodate clinical decision support functionalities

Abbreviations: ACDC, assessment curation and data collection; API, application programming interface; CT, computed tomography; EHR, electronic health record; FHIR, Fast Healthcare Interoperability Resources; HL7, Health Level Seven International; IHE, Integrating the Health Enterprise; PRO, patient-reported outcome; PROMIS, patient-reported outcomes information system; SDOH, standards for precision medicine: social determinants of health.

One such data source, patient-reported outcomes (PROs), are measurements of the outcome of a clinical intervention captured directly from the patient, often through questionnaires and without interpretation by the clinician.[57] Launched in 2017 in partnership between ONC and AHRQ, the advancing the collection of PROs through health IT project established and tested the HL7 Patient Reported Outcomes FHIR Implementation Guide while working with HL7 to ballot and approve it as a standard for trial use.[58] [59] [60] To swiftly ensure the implementation guide's applicability and value in addition to the role of a standardized API in the collection, exchange, and integration of PRO data, pilot testing was conducted by two independent organizations as a series of three development sprints, each lasting 3 to 6 months. The environment of each sprint progressed from sandbox to pre-production and then production with the goal of using electronic health records for PRO data exchange. At the end of each sprint, the feedback was used to update the implementation guide for immediate testing in the next round. Pilot testing clearly demonstrated the value of the implementation guide as a functional and foundational standard to the field, resulting in its approval as a standard for trial use by HL7 following the conclusion of the project.[58] Operations specified in the implementation guide regarding how to provide accessible tools to incorporate patient questionnaire responses into EHRs via standardized FHIR APIs have since been incorporated into the mobile device software framework SMART Markers.[61]

Recently, the Advancing Standards for Precision Medicine (ASPM) project has focused on developing standards for two data classes that were identified as high impact and prioritized for advancement to support data collection for the All of Us Research Program.[50] [62] [63] To improve the standardized collection of patient health data through mobile health applications, sensors, and wearables via the FHIR standard, the ASPM project developed and tested the HL7 FHIR Application Data Exchange Assessment Framework and Functional Requirements for Mobile Health Implementation Guide in collaboration with a large group of standards and industry stakeholders.[63] [64] Several existing standards and frameworks were adapted to meet the novel use case.[63] The resulting Framework and Implementation Guide was tested through demonstration projects both in the real world and in a sandbox environment. These small-scale implementations revealed and subsequently documented solutions to a series of technical challenges, which were reported back to the SDO partners and shared with key stakeholders. Additionally, these activities generated a series of recommendations for providers seeking to collect patient data from consumer devices following the Implementation Guide and Framework, such as ensuring the use of complete and consistent codes and terminology.[63]

The ASPM project also developed an Integrating the Healthcare Enterprise (IHE) Assessment Curation and Data Collection (ACDC) profile to establish a standards-based approach to exchange social determinants of health assessment and questionnaire data via FHIR.[65] The drafted profile built upon existing HL7 FHIR Infrastructure Work Group resources.[63] Before pilot testing, the standard was improved through Connectathons administered by IHE. A health organization in collaboration with an academic and industry partner demonstrated the use of the ACDC profile to standardize the collection of questionnaire data and subsequent reporting within a patient's electronic health record, paving the way for additional and broader collection of SDOH data for clinical care and research.[63]

In each of these projects, the ability to quickly evaluate and improve upon a newly conceived standard generated valuable feedback. The professional networks and documented lessons learned can serve as a starting point for others seeking to implement the newly established standards. The projects themselves serve as an outline for an efficient navigation of the standards development advancement process, bringing standards from conception to implementation through demonstration projects.

Demonstrating Standards to Improve Interoperability and Facilitate Clinical Data Reuse

The ONC has played a role in overseeing small-scale implementations of standardized data exchange to provide scalable resources and lessons learned ([Table 3]). The first, developing a Strategically Coordinated Registry Network (CRN) for Women's Health Technologies, was launched in 2017 as an interagency project, with ONC's role focused on developing an implementation guide to collect standardized data elements; facilitate harmonized and interoperable data exchange; and standardize the extraction and sharing of clinical data from EHR data to support the registry.[66] [67] Clinical registries can combine data from a wide range of sources, promising to reveal data-driven insights relevant for clinical care, postmarketing surveillance, quality improvement, or research, yet often they first must reveal and address gaps in should-be-standardized data collection and aggregation.[68] [69] [70] [71] First the Women's Health Technologies CRN FHIR Implementation Guide was developed under the guidance of a multi-stakeholder clinical working group and an informatics work group.[72] The implementation guide curated a harmonized set of common data elements (CDEs), including pregnancy status, preferred language, and date of procedure. Demonstration sites were recruited to test the implementation guide in the collection of its specified CDEs using standardized FHIR APIs. The two selected sites conducted three rapid-cycle sprints, each lasting ten weeks. Feedback from each sprint cycle was integrated to improve the guide. Through demonstration activities, the implementation guide was proven to aid in the establishment of a strong semantic infrastructure facilitating the exchange of the identified data elements across registries.[67] The documentation provided during pilot-testing serves as a reference for registries seeking to join the CRN, establish CRNs, or scale networks for national impact.

Table 3 The Office of the National Coordinator for Health Information Technology pilot approach to facilitate clinical data reuse through standards

	Project name
Project attributes	Coordinated Registry Network for Women's Health Technologies	Sync for Science
Project goal	Enable standardized extraction of clinical data into a coordinated registry network	Develop and demonstrate a simplified, scalable, and secure way for individuals to access and share their digital clinical data with researchers using open standards.
Standard(s)	HL7 FHIR	HL7 SMART on FHIR API OAuth 2.0 Authorization Framework
Approach	Implementation guide development and testing: Women's Health Technologies CRN Implementation Guide	Standard testing via demonstration
Use case(s)	Establish coordinated registry networks for pelvic floor disorders, uterine fibroids, and female sterilization	Patient-directed sharing of CCDS via FHIR API with the All of Us Research Program consumer application
Demonstration sites	• American Urogynecologic Society • FDA High-performance Integrated Virtual Environment; Society of Urodynamics, Female Pelvic Medicine & Urogenital Reconstruction; and New York-Presbyterian Hospital	Provider sites: • Cedars-Sinai Health System • Cerner Healthe Clinic • Duke University Health System Partners HealthCare • Rush University Medical Center • University of Missouri Health Care Health IT developers: • Allscripts (FollowMyHealth) • Cerner • eClinicalWorks • Epic
Key findings	• Infrastructure is necessary to support needed exchange and capabilities • Variance in data collection and workflow across organization affects testing and implementation • Health IT test environments are important to standards testing and development	• Health IT development required minimal effort, but provider development and testing were more resource-intensive • Participation prepared provider organizations to deploy other FHIR-based third-party apps • Implementation of different API standard versions created compatibility issues • Some data elements did not map properly (e.g., inpatient medication data)
Next steps	• Develop a common reporting framework to speed the launch and implementation of registries • Standardize data and querying capabilities using the standard • Continue to scale to enhance CRN maturity and the application of the model to multiple clinical areas • Leverage the established infrastructure for other use cases	• Expand to data types that may not be routinely collected in EHRs (e.g., claims or imaging data) • Implement standardized APIs across health IT products and provider organizations • Test additional data elements adopted under the U.S. Core Data for Interoperability

Abbreviations: API, application programming interface; CCDS, Criteria and associated Common Clinical Data Set; CRN, Coordinated Registry Network; EHR, electronic health record; FHIR, Fast Healthcare Interoperability Resources; HL7, Health Level Seven International.

The second small-scale implementation of a standardized data exchange where ONC played a role was Sync for Science pilot project, undertaken by ONC in collaboration with NIH as a public-private collaboration to establish and test patient-directed electronic health data sharing using standardized APIs.[73] Data sharing of 11 standardized data elements required within the 2015 Edition Health IT Certification Criteria and associated Common Clinical Data Set (CCDS), such as patient name, smoking status, and immunizations, was facilitated by HL7 Substitutable Medical Apps, Reusable Technology (SMART) on FHIR and OAuth 2.0 Authorization Framework (OAuth 2.0).[74] [75] [76] During the pilot project, four health IT developers were able to connect patient data from six health care provider sites with the All of Us Research Program via a consumer application. The pilot project served as an example for the 46 health centers that were enrolled to participate in the All of Us National Direct Volunteer Program and the health IT developers supporting them.[77] [78] All of the health IT developers and key subject matter experts who participated in Sync for Science also participated in the Argonaut Project, a private sector initiative managed by HL7 that aimed to accelerate the development and use of standardized APIs for patient information sharing.[79]

In both of these projects, early demonstrations laid the groundwork for more seamless onboarding of new participants. The partnerships and networks of expertise established during the pilot phase serve as an enduring resource to those in the community.

Benefits of Employing a Pilot Approach to Standards Development

In the past 5 years, ONC has matured a particular approach to standards development and testing, fueled by the rapid insights generated through pilot projects. The portfolio of ONC-led pilot projects reveals “where the rubber meets the road” by demonstrating how standards work in reality, be it in a production environment, in a health care setting, or at a research organization. The selection of practical applications that tie into real-world application or an existing process to advance standards allows pilot projects to immediately improve the use and sharing of health IT data. While projects discussed here tested HL7 and IHE standards, the approach can be applied to advance development of other health data standards maintained by these or different SDOs.

Findings from the projects often inform the progression of a standard through the standards development lifecycle from creation through balloting. For example, each phase of pilot testing under the Sync for Genes project resulted in recommendations for future versions of clinical genomics standards. The Advancing Standards Precision Medicine and the Advancing the Collection of PROs through Health IT projects were the first demonstrations of standards conceived by stakeholders for the relevant data types. The demonstrated use and lessons learned during the initial implementation of these standards were necessary to see them mature from “in development” to “balloted draft” under SDO supervision, signaling their readiness to the community and a key first step in their more widespread adoption.[80]

The demonstration of standardized data exchange during pilot projects can provide the proof of concept needed to support larger scale efforts. The use of the HL7 FHIR standard to transmit the genomic results of all routine newborn screening tests, demonstrated as part of Sync for Genes phase 2, will be leveraged by the CDC's enhancing data-driven disease detection in newborns system, which aims to be a national resource for precision public health.[81] Similarly, pilot work conducted to support the All of Us Research Program and development of the Women's Health Technologies CRN helped establish realistic expectations for the buy-in required by would-be participants to standardize their data collection activities for NIH or FDA programs and priorities. Other efforts meant to aid in implementing standards at scale such as the FHIR at Scale Taskforce may also serve as a catalyst for the next step in adoption after pilot projects.[82]

Including multiple demonstration sites within projects provides an opportunity to build informal communities of practice as implementers seek the advice and experience of others in their cohort. Pilot-testing activities often involve interdisciplinary collaboration, bringing together health IT and clinical expertise. A few months of demonstration site participation can build an enduring network of professional know-how, as participants develop expertise and thus can lead those who follow. For example, several co-chairs of the HL7 Clinical Genomics Work Group have also led past Sync for Genes pilot projects. Importantly, across projects, priorities were established based on an assessment of data needs in the field, stakeholders from the public and private sectors were engaged throughout, and all projects were conducted in collaboration with relevant SDOs—all contributing to the relevance and utility of project findings and any developed resources. Although employing this approach may be limited to institutions seeking to play a leading role in standards development, the emphasis approach places on stakeholder engagement provides numerous opportunities for community members and organizations to contribute and benefit. Furthermore, lessons from these projects are meant to advance a given standard for the benefit of the community at large and have developed approaches that may reduce barriers to participation for resource-constrained organizations.[77] [78]

Conclusion

Health IT adoption has increased across the United States in the last decade. Fully leveraging health IT and electronic health data in clinical care and research requires interoperability of systems and underlying data. Among many things interoperability depends on is the success of standards development, adoption, and use. Over the past 5 years, ONC has led efforts to advance standards development through pilot testing.

ONC's pilot approach is grounded on an understanding of data gaps and needs. By design, it engages multiple organizations across use cases and is conducted in collaboration with relevant stakeholders, including SDOs. Through its focus on multiorganization and multistakeholder engagement, the approach informally establishes communities of practice. Examples here illustrate the benefits of this approach for different data types and data exchange purposes. Each project individually and the approach generally have helped advance standards development in concert with relevant collaborators, and may offer practical guidance for provider organizations, researchers, and others interested in testing standards in their own settings. ONC will continue to advance standards through a variety of means, such as regulation, testing, coordination with relevant stakeholders, and engaging in pilot projects to rapidly understand and bridge the gap between needs for and use of standards to improve the interoperability of health data.

Clinical Relevance Statement

Increased adoption of health IT has created opportunities for research and clinical care by making electronic health data more readily available. However, leveraging those data in clinical care and research requires adoption and use of standards. Pilot testing those standards in real-world settings across different sites and for different use cases is critical. ONC is advancing pilot-testing and use of health data standards through its portfolio of projects.

Multiple Choice Questions

1. What is the typical order of activities in ONC's pilot approach?

   • Publish evaluation report, conduct landscape review, conduct pilot projects, and select approach and use cases

   • Conduct landscape review, select approach and use cases, conduct pilot projects, and publish evaluation report

   • Conduct pilot projects, conduct landscape review, select approach and use cases, and publish evaluation report

   • Select approach and use cases, publish evaluation report, conduct landscape review, and conduct pilot project

   Correct Answer: The correct answer is option b. ONC typically begins with the landscape review that helps identify research data or clinical care needs that can be met by a standard. ONC then selects an approach and use cases. The effort then moves to a pilot project and findings are summarized in an evaluation report.

2. The ONC has pilot-tested standards developed by which of the following organizations? Select all that apply.

   • Health Level 7 International

   • Integrating the Healthcare Enterprise

   • American Institute of Chemical Engineers

   • American National Standards Institute

   Correct Answer: The correct answers are options a and b. The ONC has pilot-tested the use of HL7 FHIR for different purposes, including clinical data sharing, sharing of genomic information, clinical registry data, and patient-reported outcomes. ONC has also pilot-tested the use of an IHE profile to share social determinants of health data.

3. For what purposes has ONC advanced the development of testing of health data standards?

   • Present and share clinical genetic data

   • Facilitate clinical data reuse

   • Integrate social and patient-generated health data into the EHR

   • All of the above

   Correct Answer: The correct answer is option d. The ONC has advanced standards to enable sharing of different kinds of data including clinical, genetic, and social and patient-generated health data for use in clinical care, research, and other purposes.

Conflict of Interest

None declared.

Acknowledgments

The authors would like to thank Carrie Milligan for copy-editing support, Donald Harris Jr. for graphics support, and Viet Nguyen, MD, Stratametrics LLC for his valuable advice on this document. They would also like to thank Kevin Chaney, Stephanie Garcia, and Tracy Okubo from the Office of the National Coordinator for Health Information Technology, demonstration sites, and other federal and private sector collaborators for leadership and contributions on the pilot projects summarized in this article.

Protection of Human and Animal Subjects

There were no human and/or animal subjects included in this project.

Note

The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the U.S. Department of Veterans Affairs.

Authors' Contributions

A.F.D., T.Z.C., and P.J.W. led the conception of the article. All authors revised the article critically and provided intellectual content, and they approved the final version for submission. The order of authors listed in the manuscript has been approved by all authors.

• References

• 1 Henry J, Pylypchuk Y, Searcy T. et al. Adoption of electronic health record systems among U.S. non-federal acute care hospitals: 2008–2015. Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/briefs/2015_hospital_adoption_db_v17.pdf
• 2 Office of the National Coordinator for Health Information Technology. Office-based physician electronic health record adoption, health IT QUICK-STAT #50. Accessed February 8, 2021 at: https://dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php
• 3 Kruse CS, Beane A. Health information technology continues to show positive effect on medical outcomes: systematic review. J Med Internet Res 2018; 20 (02) e41
• 4 McCullough JS, Casey M, Moscovice I, Prasad S. The effect of health information technology on quality in U.S. hospitals. Health Aff (Millwood) 2010; 29 (04) 647-654
• 5 Agha L. The effects of health information technology on the costs and quality of medical care. J Health Econ 2014; 34: 19-30
• 6 Jones SS, Rudin RS, Perry T, Shekelle PG. Health information technology: an updated systematic review with a focus on meaningful use. Ann Intern Med 2014; 160 (01) 48-54
• 7 Alotaibi YK, Federico F. The impact of health information technology on patient safety. Saudi Med J 2017; 38 (12) 1173-1180
• 8 Chaudhry B, Wang J, Wu S. et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; 144 (10) 742-752
• 9 Sadoughi F, Nasiri S, Ahmadi H. The impact of health information exchange on healthcare quality and cost-effectiveness: a systematic literature review. Comput Methods Programs Biomed 2018; 161: 209-232
• 10 Hersh WR, Totten AM, Eden KB. et al. Outcomes from health information exchange: systematic review and future research needs. JMIR Med Inform 2015; 3 (04) e39
• 11 Lehne M, Sass J, Essenwanger A, Schepers J, Thun S. Why digital medicine depends on interoperability. NPJ Digit Med 2019; 2: 79
• 12 Perlin JB. Health information technology interoperability and use for better care and evidence. JAMA 2016; 316 (16) 1667-1668
• 13 Interoperability in Healthcare. Accessed April 19, 2021 at: https://www.himss.org/resources/interoperability-healthcare
• 14 Haug PJ, Narus SP, Bledsoe J, Huff S. Promoting national and international standards to build interoperable clinical applications. AMIA Annu Symp Proc 2018; 2018: 555-563
• 15 Chen ES, Melton GB, Sarkar IN. Translating standards into practice: experiences and lessons learned in biomedicine and health care. J Biomed Inform 2012; 45 (04) 609-612
• 16 Office of the National Coordinator for Health Information Technology. About ONC (February 14, 2019). Accessed February 8, 2021 at: https://www.healthit.gov/topic/about-onc
• 17 Office of the National Coordinator for Health Information Technology. Scientific initiatives. (September 25, 2020). Accessed February 8, 2021 at: https://www.healthit.gov/topic/scientific-initiatives
• 18 Zayas-Cabán T, Chaney KJ, Rucker DW. National health information technology priorities for research: a policy and development agenda. J Am Med Inform Assoc 2020; 27 (04) 652-657
• 19 Zayas-Cabán T, Abernethy AP, Brennan PF. et al. Leveraging the health information technology infrastructure to advance federal research priorities. J Am Med Inform Assoc 2020; 27 (04) 647-651
• 20 Office of the National Coordinator for Health Information Technology. National health IT priorities for research. (January 15, 2020). Accessed February 8, 2021 at: https://healthit.gov/research-agenda
• 21 Zayas-Cabán T, Wald JS. Opportunities for the use of health information technology to support research. JAMIA Open 2020; 3: 321-325
• 22 Richesson RL, Krischer J. Data standards in clinical research: gaps, overlaps, challenges and future directions. J Am Med Inform Assoc 2007; 14 (06) 687-696
• 23 Bouhaddou O, Cromwell T, Davis M. et al. Translating standards into practice: experience and lessons learned at the Department of Veterans Affairs. J Biomed Inform 2012; 45 (04) 813-823
• 24 Rozwell C, Kush R, Helton E. Saving time and money. Appl Clin Trials 2007; 16: 70-74
• 25 Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood) 2009; 28 (02) w282-w293
• 26 Bassi J, Lau F. Measuring value for money: a scoping review on economic evaluation of health information systems. J Am Med Inform Assoc 2013; 20 (04) 792-801
• 27 International Organization for Standardization. Economic benefits of standards. Accessed April 7, 2021 at: https://www.iso.org/files/live/sites/isoorg/files/archive/pdf/en/ebs_case_studies_factsheets.pdf
• 28 Kilbourne AM, Goodrich DE, Miake-Lye I, Braganza MZ, Bowersox NW. Quality enhancement research initiative implementation roadmap: toward sustainability of evidence-based practices in a learning health system. Med Care 2019; 57 (10, Suppl 10 Suppl 3): S286-S293
• 29 Kötter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators--a systematic review. Implement Sci 2012; 7: 21
• 30 Ratwani R, Fairbanks T, Savage E. et al. Mind the Gap. A systematic review to identify usability and safety challenges and practices during electronic health record implementation. Appl Clin Inform 2016; 7 (04) 1069-1087
• 31 Office of the National Coordinator for Health Information Technology. ONC change package for improving EHR usability (February 2018). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/playbook/pdf/usability-change-plan.pdf
• 32 DeBar LLJJ, Tuzzio L, Vazquez MA. Assessing feasibility: spotlight on demonstration projects. Accessed February 8, 2021 at: https://rethinkingclinicaltrials.org/chapters/conduct/assessing-feasibility/spotlight-on-four-demonstration-projects
• 33 Fiordalisi C, Borsky A, Chang S, Guise JM. AHRQ EPC series on improving translation of evidence into practice for the learning health system: introduction. Jt Comm J Qual Patient Saf 2019; 45 (08) 558-565
• 34 Guterman S, Davis K, Stremikis K, Drake H. Innovation in Medicare and Medicaid will be central to health reform's success. Health Aff (Millwood) 2010; 29 (06) 1188-1193
• 35 Nevedal AL, Reardon CM, Jackson GL. et al. Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study. Implement Sci Commun 2020; 1: 61
• 36 Forrow S, Campion DM, Herrinton LJ. et al. The organizational structure and governing principles of the Food and Drug Administration's Mini-Sentinel pilot program. Pharmacoepidemiol Drug Saf 2012; 21 (Suppl. 01) 12-17
• 37 Kilbourne AM, Jones PL, Atkins D. Accelerating implementation of research in learning health systems: lessons learned from VA health services research and NCATS clinical science translation award programs. J Clin Transl Sci 2020; 4 (03) 195-200
• 38 Matney SA, Heale B, Hasley S. et al. Lessons Learned in creating interoperable fast healthcare interoperability resources profiles for large-scale public health programs. Appl Clin Inform 2019; 10 (01) 87-95
• 39 Office of the National Coordinator for Health Information Technology. Precision medicine. (June 23, 2020) Accessed February 8, 2021 at: https://www.healthit.gov/topic/scientific-initiatives/precision-medicine
• 40 Office of the National Coordinator for Health Information Technology. Building data infrastructure to support patient-centered outcomes research (PCOR) (March 31, 2020). Accessed February 8, 2021 at: https://www.healthit.gov/pcor
• 41 Zayas-Cabán T, Chaney KJ, Rogers CC, Denny JC, White PJ. Meeting the challenge: Health information technology's essential role in achieving precision medicine. J Am Med Inform Assoc 2021; ocab032
• 42 Office of the National Coordinator for Health Information Technology. Health IT playbook. Accessed February 8, 2021 at: https://www.healthit.gov/playbook/sdo-education/chapter-2/
• 43 Connectathon IHE. A unique testing opportunity. Accessed April 7, 2021 at: https://www.ihe.net/participate/connectathon/
• 44 HL7 FHIR Connectathon FAQs. Accessed February 8, 2021 at: https://confluence.hl7.org/display/FHIR/HL7+FHIR+Connectathon+FAQs
• 45 Schulz S, Stegwee R, Chronaki C. Standards in Healthcare Data. In: Kubben P, Dumontier M, Dekker A. eds. Fundamentals of Clinical Data Science. Cham, Switzerland: 2019: 19-36
• 46 Proctor E, Silmere H, Raghavan R. et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011; 38 (02) 65-76
• 47 Office of the National Coordinator for Health Information Technology. Sync for genes. . (January 7, 2021). Accessed February 8, 2021 at: https://www.healthit.gov/topic/sync-genes
• 48 White PJ, Halamka J. Precision medicine task force transmittal letter. (September 25, 2015). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/facas/PMTF_Transmittal_Letter_2015-09-25_v2.pdf
• 49 Gallagher L, Malec A. Health information technology standards committee transmittal letter. (July 22, 2016). Accessed April 20, 2021 at: https://www.healthit.gov/sites/default/files/facas/HITSC_PMI_Phase2_Recs_2016-07-22.pdf
• 50 Denny JC, Rutter JL, Goldstein DB. et al; All of Us Research Program Investigators. The “all of us” research program. N Engl J Med 2019; 381 (07) 668-676
• 51 Office of the National Coordinator for Health Information Technology. Sync for genes: enabling clinical genomics for precision medicine via HL7®fast healthcare interoperability resources®. (November 28, 2017). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/sync_for_genes_report_november_2017.pdf
• 52 Genetic Reporting Implementation Guide. (July 25, 2018). Accessed February 8, 2021 at: http://hl7.org/fhir/uv/genomics-reporting/
• 53 HL7 domain analysis model: clinical genomics, release1. Accessed February 8, 2021 at: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=479
• 54 Office of the National Coordinator for Health Information Technology. Sync for genes phase 2 project: exploring approaches to make clinical genomics available at the point-of-care final report. (April 2020). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/page/2020-04/Sync%20for%20Genes%202%20Final%20Report.pdf
• 55 Office of the National Coordinator for Health Information Technology. Sync for genes phase 3 engaging laboratories final report. (January 2021). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/page/2021-01/Sync-for-Genes-Phase-3-Engaging-Laboratories.pdf
• 56 Garcia SJ, Zayas-Cabán T, Freimuth RR. Sync for genes: making clinical genomics available for precision medicine at the point-of-care. Appl Clin Inform 2020; 11 (02) 295-302
• 57 U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes 2006; 4: 79
• 58 Office of the National Coordinator for Health Information Technology. Advancing the collection and use of patient-reported outcomes through health information technology. (March 2020). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/page/2020-03/ONCPROFinalReportFinal.pdf
• 59 Patient Reported Outcomes FHIR Implementation Guide. (Aug 31, 2018). Accessed February 8, 2021 at: http://www.hl7.org/fhir/us/patient-reported-outcomes/2018Sep/
• 60 Office of the National Coordinator for Health Information Technology. Patient-reported outcomes through health IT project. (March 27, 2020). Accessed February 8, 2021 at: https://www.healthit.gov/topic/scientific-initiatives/pcor/patient-reported-outcomes-through-healthit-pro
• 61 Sayeed R, Gottlieb D, Mandl KD. SMART Markers: collecting patient-generated health data as a standardized property of health information technology. NPJ Digit Med 2020; 3: 9
• 62 Office of the National Coordinator for Health Information Technology. Advancing standards for precision medicine. (January 26, 2021). Accessed February 8, 2021 at: https://www.healthit.gov/topic/advancing-standards-precision-medicine
• 63 Office of the National Coordinator for Health Information Technology. Advancing standards for precision medicine final report. (January 2021). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/page/2021-01/Advancing-Standards-in-Precision-Medicine.pdf
• 64 Application Data Exchange Assessment Framework and Functional Requirements for Mobile Health. (February 18, 2020). Accessed February 8, 2021 at: https://hl7.github.io/fhir-project-mhealth/index.html
• 65 IHE Patient Care Coordination Technical Framework Supplement – Assessment Curation and Data Collection (ACDC). (March 24, 2020). Accessed February 8, 2021 at: https://www.ihe.net/uploadedFiles/Documents/PCC/IHE_PCC_Suppl_ACDC.pdf
• 66 Office of the National Coordinator for Health Information Technology. Coordinated Registry Network for Women's Health Technologies. (June 14, 2020). Accessed February 8, 2021 at: https://www.healthit.gov/topic/scientific-initiatives/pcor/coordinated-registry-network-womens-health-technologies-crn
• 67 Office of the National Coordinator for Health Information Technology. Developing a strategically coordinated registry network (CRN) for women's health technologies final report. (June 2020). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/page/2020-06/Strategically-CRN-for-Womens-Health-Technologies.pdf
• 68 Blumenthal S. The use of clinical registries in the united states: a landscape survey. EGEMS (Wash DC) 2017; 5 (01) 26
• 69 U.S. Food and Drug Administration. FDA facts: postmarket patient registry ensures access to safe and effective devices. (December 21, 2017). Accessed February 8, 2021 at: https://www.fda.gov/about-fda/innovation-fda/fda-facts-postmarket-patient-registry-ensures-access-safe-and-effective-devices
• 70 Agency for Healthcare Research and Quality. Registries for evaluating patient outcomes: a user's guide. (February 2018). Accessed February 8, 2021 at: https://effectivehealthcare.ahrq.gov/sites/default/files/registries-guide-3rd-ed-addendum-research-2018.pdf
• 71 Workman TA. Engaging Patients in Information Sharing and Data Collection: The Role of Patient-Powered Registries and Research Networks [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013. Sep. Report No.: AHRQ 13-EHC124-EF. PMID: 24156118
• 72 Women's Health Technologies (WHT) Coordinated Registry Network. (CRN) FHIR implementation guide. (March 26, 2019). Accessed February 8, 2021 at: https://build.fhir.org/ig/HL7/coordinated-registry-network/
• 73 Office of the National Coordinator for Health Information Technology. Sync for science. (November 12, 2020). Accessed February 8, 2021 at: https://www.healthit.gov/topic/sync-science
• 74 Office of the National Coordinator for Health Information Technology. Sync for science pilot project: participants' experiences. (November 2020). Accessed February 8, 2021 at: https://www.healthit.gov/sites/default/files/page/2020-11/Sync%20for%20Science%20Pilot%20Project_Participants%20Experiences.pdf
• 75 Office of the National Coordinator for Health Information Technology, Department of Health and Human Services, 2015 Edition Health Information Technology (Health IT) Certification Criteria, 2015 Edition Base Electronic Health Record (EHR) Definition, and ONC Health IT Certification Program Modifications. 45 C.F.R §170. Accessed February 8, 2021 at: https://pubmed.ncbi.nlm.nih.gov/26477063/
• 76 Office of the National Coordinator for Health Information Technology. 2015 edition common clinical data set (CCDS) reference document. (February 22, 2018). Accessed April 19, 2021 at: https://www.healthit.gov/sites/default/files/ccds_reference_document_v1_1.pdf
• 77 Health Resources and Services Administration. Fiscal Year 2018 Advancing Precision Medicine Supplemental Funding Opportunity (APM). Accessed February 8, 2021 at: https://allofus.nih.gov/sites/default/files/final_hrsa-18-126_apm_supplement.pdf
• 78 Health Resources and Services Administration. FY 2018 Advancing precision medicine (APM) supplemental awards. (May 2020). Accessed February 8, 2021 at: https://bphc.hrsa.gov/program-opportunities/advancing-precision-medicine-fy2018-awards
• 79 About the Argonaut Project. (August 31, 2020). Accessed April 19, 2021 at: https://argonautwiki.hl7.org/Main_Page
• 80 Office of the National Coordinator for Health Information Technology. Interoperability standards advisory structure. Accessed April 7, 2021 at: https://www.healthit.gov/isa/isa-structure
• 81 Jones D, Garcia S, Ruiz-Schultz N. et al. A strong start: enhancing newborn screening for precision public health. (October 13, 2020). Accessed February 8, 2021 at: https://blogs.cdc.gov/genomics/2020/10/13/a-strong-start/
• 82 FHIR at Scale Taskforce (FAST). (December 22, 2020). Accessed February 8, 2021 at: https://oncprojectracking.healthit.gov/wiki/pages/viewpage.action?pageId=43614268

Publication History

Received: 20 February 2021

Accepted: 20 May 2021

Article published online:
11 August 2021