We Know What You Agreed To, Don't We?—Evaluating the Quality of Paper-Based Consents Forms and Their Digitalized Equivalent Using the Example of the Baltic Fracture Competence Centre Project

Abstract

Introduction The informed consent is the legal basis for research with human subjects. Therefore, the consent form (CF) as legally binding document must be valid, that is, be completely filled-in stating the person's decision clearly and signed by the respective person. However, especially paper-based CFs might have quality issues and the transformation into machine-readable information could add to low quality. This paper evaluates the quality and arising quality issues of paper-based CFs using the example of the Baltic Fracture Competence Centre (BFCC) fracture registry. It also evaluates the impact of quality assurance (QA) measures including giving site-specific feedback. Finally, it answers the question whether manual data entry of patients' decisions by clinical staff leads to a significant error rate in digitalized paper-based CFs.

Methods Based on defined quality criteria, monthly QA including source data verification was conducted by two individual reviewers since the start of recruitment in December 2017. Basis for the analyses are the CFs collected from December 2017 until February 2019 (first recruitment period).

Results After conducting QA internally, the sudden increase of quality issues in May 2018 led to site-specific feedback reports and follow-up training regarding the CFs' quality starting in June 2018. Specific criteria and descriptions on how to correct the CFs helped in increasing the quality in a timely matter. Most common issues were missing pages, decisions regarding optional modules, and signature(s). Since patients' datasets without valid CFs must be deleted, QA helped in retaining 65 datasets for research so that the final datapool consisted of 840 (99.29%) patients.

Conclusion All quality issues could be assigned to one predefined criterion. Using the example of the BFCC fracture registry, CF-QA proved to significantly increase CF quality and help retain the number of available datasets for research. Consequently, the described quality indicators, criteria, and QA processes can be seen as the best practice approach.

Keywords

consent mechanisms - GDPR - informed consent - quality assurance

Introduction

Due to the EU-General Data Protection Regulation (EU-GDPR),[1] European countries and research facilities face the challenge of implementing and using consent mechanisms and workflows. Especially, in medical research, informed consent (IC) is a main requisite: The Declaration of Helsinki[2] states that each participant in a clinical trial must give an IC. Article 9 of the EU-GDPR[1] supports this in defining medical and person-identifying data as special categories of personal data that can only be processed based on an IC. Consequently, research projects (except ones with legal bases like national [cancer] registries) have to ensure that for every study participant a valid informed consent form (CF) was collected. Since a CF is the base for research in compliance with data protection regulations, it should reflect and, thus, protect the participant's rights and will. This means, the CF must (1) enable the participant to express her/his will for participating in specific study tasks and sharing data for specific purposes and (2) be completely filled in, including the participant's signature. A major problem for health care research is that datasets with incorrect CFs cannot be used for research and must even be permanently deleted if the quality issues are not corrected. This loss of research data as basis for innovation and improvements in medical care due to an incorrectly completed form is avoidable. Especially in developing registries with small numbers of datasets and participants, this leads to unnecessary further reductions in evaluable data for medical research and the scientific community. For example, Vogele et al found that in 1,424 paper CFs only (1) 96.5% had all necessary signatures, that is, created legal certainty for medical data capture and (2) 77.9% were sufficiently well completed, that is, initially correct.[3] Consequently, datasets based on more than 20% of the CFs would be needlessly lost to research.

The Baltic Fracture Competence Centre (BFCC) project aims at improving fracture care with partners in seven Baltic countries. BFCC includes a transnational fracture registry, which builds the base for innovations to improve fracture diagnosis and treatment and reduce complications.[4] Since data protection guidelines recommend the separation of (1) identifying and (2) medical data to prevent reidentification of patients,[5] a Trusted Third Party (TTP)[6] is used to manage person-identifying data within the BFCC project. As stated by Good Clinical Practice (GCP),[7] monitoring of a study's data should add to protecting patients' rights. However, in most cases, only random samples are examined, which means that only individual medical datasets including the CF are quality assured. Quality assurance (QA) measures to ensure the quality of study-specific forms usually encompasses on-site visits or monitoring with source data verification (SDV) of a defined sample.[8] [9] [10] [11] [12] However, using paper-based CFs only and conducting such a QA is impractical in an international endeavor like the BFCC registry, and thus, a digital mapping of CFs to enable central data management and QA is implemented. Additionally, such machine-readable information can also be used for automatic access to consent states for all involved systems like hospital, imaging, or laboratory information systems. Therefore, the BFCC registry conducted CF-QA measures to ensure the existence of CFs in compliance with regulations like the EU-GDPR.[1]

Quality issues in consent processes including electronic data capture could arise in two forms: first, the paper-based CF is incompletely filled in or contains errors including ambiguous answers, for example, both boxes, “yes” and “no,” are ticked for a module like “I want to take part in additional examinations.” Second, the manual data entry of paper-based information into an electronic system leads to additional errors like typos or ticking the wrong box in the digital form. To prevent errors arising from the second error source, QA using SDV is necessary to ensure the correct representation of the paper-based patient's will in electronic systems. A literature search in PubMed in May 2022 using the search terms (“informed consent” OR “consent form” OR consent) AND (“patient registry” OR “patient registries”) AND (quality OR “data quality” OR “quality control” OR “quality assurance”) including MeSH terms revealed that there is no published best practice example for CF-QA in a transnational context.

The aim of this paper is to (1) evaluate the quality of paper-based CFs of the first data collection phase for the BFCC registry, (2) to determine the quality of the digitalized CFs, and (3) describe best practices in CF-QA. Therefore, this paper analyses the following questions using the example of the BFCC registry:

1. Do quality issues arise in using uniform paper-based CFs?

2. Do QA measures including feedback improve the initial quality of paper-based CF collection by reducing quality issues?

3. Does manual data entry lead to a significant error rate in digitalizing paper-based CFs?

Methods

Design of the Baltic Fracture Competence Centre Project and Registry

The BFCC project was a cooperation of 13 organizations from seven countries around the Baltic Sea to foster innovation within fracture management. Therefore, fracture registry-sites at four hospitals around the Baltic Sea Region were established to collect fracture treatment data in one transnational fracture registry. Patients of full age (18+ years) admitted to a participating hospital could be recruited for the BFCC project, when they were diagnosed with at least one fracture resulting in a hospital stay for treatment (inpatient), and filled in and signed a written CF prior to participation.

Before a registry site could start recruiting patients, a positive vote from the local ethics committee needed to be received and sent to the TTP. Additionally, the initial training was conducted before clinical staff could enter data.

Obtaining Consent Form for the Baltic Fracture Competence Centre Registry

To obtain a patient's consent, a paper-based, BFCC-wide uniform CF was used for the BFCC registry. The use of paper-based CFs was due to limited hardware (e.g., SignPads) as well as the unplanned nature of fractures and the case dependence, individually resulting processes in the participating hospitals. For example, the paper-based CFs were laid out at every ward, which could be involved in (unplanned) fracture treatment, in one clinical workflow. Then, the doctor or nurse on duty tried—if possible—to recruit a fracture patient for the BFCC registry.

To enable patients to understand the CF and avoid language barriers, the CF was translated into and, thus, available in the country-specific language. Consequently, CFs were available in Estonian, German, Lithuanian, Polish, and, additionally, English. After BFCC clinical staff informed the patient about the project and answered all questions, the patient or its guardian—if necessary, with help of the clinical staff—had to fill in the paper-based BFCC CF. Only completely filled-in and signed CFs were valid.

To digitize and manage the paper-based CFs, the generic Informed Consent Service (gICS) of the University Medicine Greifswald was used.[13] Using gICS, paper-based CFs were entered into an electronic Case Report Form of the TTP with the look-and-feel of the BFCC CF (English version) by clinical staff and automatically, digitally mapped into granular electronic policies within the BFCC project. Such an electronic mapping of paper-based information enables automated checks and queries between systems to determine if the patient consented to certain processes and modules (e.g., additional examinations). Concluding the data entry, the paper-based CF was scanned and uploaded as an attachment to the electronic CF, allowing for central QA and SDV. One exception was CFs from Polish registry sites. Those registry sites did not transfer CFs due to local regulations and, consequently, had to locally assure the quality of their CFs without central QA.

Structure of the Uniform Baltic Fracture Competence Centre Consent Form

A description on requirements regarding the development of an CF complying to legal requirements can be derived from the EU-GDPR1 and publications from the Technology, Methods, and Infrastructure for Networked Medical Research e. V. (TMF).[5] Based on this information, the CF content was developed in multiple workshops between the BFCC partners, that is, by clinical, scientific, and TTP staff. The uniform BFCC CF was originally provided in German and English only, with the English version being the template for local translations to reduce language barriers for patients. The CF in the local language was part of the positive vote from the local ethics committee.

The BFCC CF included an introductory text with information about the planned scope of the BFCC project and registry, four sections with optional modules, which the patient can consent to or decline, followed by information on the right to withdraw the consent at any time (including how to withdraw and where to send the withdrawal to) and other rights of the patient. Finally, the last section asked the participant—or a legal guardian—as well as the clinical staff member, who informed the patient about the BFCC project, its registry and led through the consent process, to sign the CF and provide the date of signatures.

All modules, to which a patient can consent to, were written from the patient's perspective, that is, “I grant my consent to […]” to make clear to patients that it's their personal decision. All optional module sections had two answer options, which needed to be ticked either “yes” or “no.” This was important to unambiguously show the patient's decision.

The first optional module section informed on how fracture treatment data were collected, stored, and used and asked for the patient's consent to the storage and use of treatment data. This module was mandatory. If a patient declined to consent to the storage and use of treatment data, the patient could not be included in the BFCC project and no further data were collected.

The second optional module was only relevant for patients in Scandinavian countries to ask for permission to use the national (health) insurance number or personal identity number. In the end, this module became irrelevant because no Scandinavian patients were recruited for the BFCC registry.

The last optional module asked the patients to allow recontacting. Since it is possible that in the future new scientific, clinical, or device-related questions will arise, the hospital may want to invite BFCC participants to participate in supplemental surveys, studies, or follow-up examinations as necessary.

Finally, the patient needed to provide the name of the treating hospital (if not prefilled), his/her name, date of signature, and sign the CF. Afterward, the informing clinical staff member needed to state his/her name and sign the CF.

Quality Criteria for Consent Forms in the Baltic Fracture Competence Centre Registry

Unlike most studies using SDV, the QA of BFCC CFs is not limited to random samples only. Within the BFCC registry, SDV was used for each BFCC CF to safeguard patients' rights. This means that all digitized CFs were centrally verified using the scans of the paper-based origin—with only Polish registry-sites as exception.

Due to its sensitive and legally binding nature, strict criteria have to determine whether a CF is valid or not. To determine the quality criteria and indicators and to develop the BFCC QA concept, workshops and bilateral discussions were conducted with the BFCC data management coordinator, data protection experts, TTP staff, and staff from participating hospitals. Thus, several meetings were held to identify best practices in QA of CFs, which shaped the conceptual design. Constant evaluation of the concept during the project's first data capture phase (2017–2019) led to adaptations regarding the QA concept and the subsequent processes including giving feedback to recruiting hospitals.

Following Nonnemacher et al,[8] the indicators “completeness” (TMF-1046) and “correctness” (TMF-1045), which is the “consistency of registry data with original data related to observation units,” were selected as higher-level quality indicators. Additionally, a third indicator “legal certainty” was defined by the TTP to comply with the above-mentioned regulations like GCP and EU-GDPR.[1] [2] [7]

For the indicator “completeness” QA checked whether the CF was completely filled in, that is, all obligatory items are filled in, and all data have been recorded digitally as well as all pages of a CF are scanned and available in a digital form to the TTP. For the indicator “correctness,” it was verified that the electronic version(s) of a patient's CF(s) correspond to those recorded on paper, especially the stated patient's decisions regarding optional modules. The indicator “legal certainty”[1] [2] [7] determined that the date of signature as well as the signatures of the participant or its guardian, and the informing clinical staff are given on each paper-based CF.

To conduct comprehensive QA measures, specific criteria were defined for each higher-level indicator. The criteria stated specific categories of quality issues and were used as a checklist to determine the validity and quality of each recorded CF. [Table 1] shows the resulting criteria for QA in CFs for the BFCC project.

Table 1 BFCC criteria for quality assurance and possible categories of quality issues including their description

Higher-level indicator	BFCC criteria	Categories of quality issues	Description
Completeness	Obligatory items are filled in	Missing obligatory items	Missing obligatory items, e.g., first/last name, date of signature
Completeness	Use of option boxes (to tick “yes” or “no” regarding to patient's will) on paper-based form	Missing ticked boxes of optional modules	At least one option box was not unambiguously ticked “yes” or “no.” The patient's will is unknown.
Completeness	Use of option boxes (to tick “yes” or “no” regarding to patient's will) was digitally recorded	Missing ticked boxes of optional modules	At least one option box was not unambiguously ticked “yes” nor “no.” The patient's will is unknown.
Completeness	Upload of CF scan including all CF pages	(a) No CF scan uploaded	(b) Incomplete scan
		(a) No CF scan was uploaded	(b) Not all CF pages were included in the scan (i.e., missing pages)
Correctness	Use of valid project-specific CF version	No valid CF version	An unknown CF was used instead of the BFCC-wide uniform CF (V1.0.1–1.2.0)[a].
Correctness	Electronic CF (including optional modules) correspond to paper-based origin	Consent of optional modules is wrongly digitized	At least one option box was originally ticked as “yes” but digitally recorded as “no” or the other way round.
Legal certainty	A complete CF scan belonging to the respective patient is uploaded	No, wrong or incomplete CF scan uploaded	No CF scan or a CF scan with missing pages or a CF scan belonging to a different patient was uploaded.
Legal certainty	All signatures are given on CF including signature date	Missing signature(s)	The patient and/or the clinical staff (e.g., treating doctor) has/have not signed the CF.

Abbreviations: BFCC, Baltic Fracture Competence Centre; CF, consent form.

^a Current Consent Form (CF) Version: 1.2.0. Older BFCC CF-versions are still valid for patients, which were recruited with the respective older CF-version, e.g., before the new CF-version was distributed.

Quality Assurance and Data Analysis

The software tool gICS,[13] which provides a web interface for managing CFs and withdrawals, was used to perform QA. Thus, TTP staff can assess (1) which CFs have been electronically created for a BFCC participant and can verify that (2) a scan of the paper-based document has been uploaded for each electronically created CF, (3) the scan is complete, (4) the paper-based form is completely and correctly filled in, and (5) the electronic equivalent is complete and concordant to the paper-based origin. As part of the QA process, a feedback structure, that is, reporting, was also implemented for the BFCC project.

QA was conducted by two independent reviewers checking all CFs including SDV, that is, all CFs were assessed by TTP staff frequently at the end of each month. One exception is CFs from Polish registry sites as mentioned before. Nevertheless, Polish BFCC participants are included in the following data analyses as having completed QA without quality issues according to the registry sites local QA.

Starting in June 2018, a monthly site-specific report was sent to the respective hospital. In case of quality issues, TTP tickets were created (starting in August 2018) and assigned to the respective clinical staff. TTP tickets enabled clinical staff to upload CF scans of corrected paper-based forms directly to the TTP via encrypted, one-to-one connection.

Consequently, the basis for the following analyses is the CF-QA reports of the BFCC project from June 2018 until February 2019 (end of first recruitment period). Outcomes are completed CFs with/without quality issues and identified quality issues.

Since all CFs are quality assured, the result of the completed QA processes represented the data pool of the first recruitment period of the BFCC project.

Results

The recruitment and data entry of the BFCC project started with the first registry site on December 19, 2017. Subsequently, further registry sites started recruiting in the following months. During the first data collection phase from December 2017 until February 2019, a total of 846 patients filled in paper-based CFs and were digitally registered for the BFCC registry with an average of approximately 78 patients per month (see [Fig. 1]).

Fig. 1 Total numbers of consented BFCC patients for the BFCC fracture registry.

All but two patients consented to all optional modules. In both cases, the patients indicated their refusal by ticking “no.” However, in both cases, clinical staff manually entered “yes” for the refused modules when digitizing the CF. QA identified both cases and processed it on a case-by-case basis as described in the following sections.

The Need to Introduce Monthly Quality Assurance Reports

At first, CF QA checks within the BFCC were conducted TTP-internally on a regular basis at the end of each month without reporting the results. In May 2018, a significant increase in quality issues was noticed: QA detected quality issues in 42 of 272 CFs. This means that more than 15% of the CFs received by TTP were not initially valid for research. Before, the percentage was below 2%. Consequently, it was decided to send monthly QA reports to the hospitals and, if required, offer additional follow-up training.

The first site-specific quality reports were sent on June 27, 2018. In this first report, only 239 CFs (81.57%) of the 293 CFs met all quality criteria. Of the 54 ICs with quality issues, the result was as follows:

• Three CFs were not uploaded as a scan at all,

• Ten CFs had three quality issues (incomplete scan, missing ticked boxes of optional modules, and missing signatures),

• 20 CFs had two quality issues:

  • (a) incomplete scan and missing signatures (n = 15)

  • (b) incomplete scan and missing ticked boxes of optional modules (n = 2)

  • (c) missing ticked boxes of optional modules and missing signature(s) (n = 3)

• 21 CFs had only one quality issue:

  • (a) missing signature(s) (n = 6)

  • (b) missing ticked boxes of optional modules (n = 15)

It has to be noted that an incomplete scan, that is, missing pages in the uploaded scan, might automatically lead to the issues “missing ticked boxes of optional modules” or “missing signature(s).” This was the case when the missing pages contained the optional modules or signature fields, respectively.

After the first CF-QA-reports, the CF quality increased significantly (see [Fig. 2]) and, if possible, the CFs with quality issues were corrected. In case a correction of the CF was not possible, because the patient already left the hospital and could not be recontacted, the CF and all patient data were deleted from the BFCC databases.

Fig. 2 Consent form quality in the BFCC project over time.

In July 2018—after the first report and follow-up training were conducted—only nine new CFs had quality issues. Due to corrections, the number of CFs with quality issues dropped to a total of 12 in August 2018. In the following months, quality issues occurred only in one to three new CFs per month.

Most commonly, CF pages were missing in the scan or the patient had not ticked the boxes of all optional modules according to her/his decision. In CF scans with missing pages, mostly pages with even page numbers were missing. This was caused by the two-sided printed form, which was scanned only one-sided.

Frequently, missing signature(s) (of informing clinical staff, patient, or both) had to be reported.

Adapting Quality Assurance to Systematic Quality Issues and Implementing the Final Quality Assurance Checklist

At first, QA checked whether all items were filled in the respective fields. However, one systematic issue in a translated CF language version was that patients signed in the “date of signature” field. Above the two fields “date” and the subsequent “signature” was the subheading “signature,” which obviously mislead patients. Consequently, the quality criteria were adapted so that it was irrelevant for CF-QA if the signature was in the correct “signature” field or in the “date of signature” field. Furthermore, scanned pages were sometimes in the wrong order or upside down. Consequently, the order or orientation of the scanned pages were irrelevant for CF-QA as long as all pages were included in the uploaded scan and readable. Additionally, optional modules were refused or consented by ticking boxes for “yes” or “no.” For QA it was negligible, whether the boxes were perfectly check marked or colored-in as long as the choice was without ambiguity. Furthermore, the CF-QA raised awareness that changes in CF versions were implemented with a temporal delay due to the paper-based clinical workflows, in which the paper-based CFs were laid out at every fracture-related ward. This led to scenarios where (1) old CF versions were still used until the supply of printed forms was exhausted, or (2) old as well as new versions were used at a hospital depending on ward or clinical staff. Additionally, new CF versions needed to be translated into the local language, which automatically led to time lags. Thus, invalidating old CF versions as soon as new versions are distributed was considered impractical. As a consequence, the BFCC partners agreed on considering all CF versions valid. With each new version were new electronic policies implemented. Due to this, QA needed to make sure that the electronic equivalent only had the policies activated, which were included in the corresponding CF version.

In summary, the final QA checklist consisted of eight questions (see [Fig. 3]). If at least one question was answered “no,” the CF was considered invalid and included in the monthly site-specific feedback report with a need for correction. All corrections were also checked against the checklist and a CF was classified as invalid as long as not all questions could be answered with “yes.”

Fig. 3 Final checklist for BFCC quality assurance of consent forms.

One-Time Quality Issues

QA also identified noteworthy one-time issues: Once the same CF-scan for two time-wise consecutive patients was uploaded, invalidating one patient's dataset. This was easily resolved after sending the feedback report with specifying the criterion “missing CF scan” in detail to the hospital, which uploaded the correct CF-scan for the second patient.

Additionally, two CFs had signatures consisting of slightly more than one straight line, which led to an enquiry by TTP regarding potentially “missing signatures.” For both cases, the treating hospitals declared that this was the patient's valid signature due to physical impairments.

The most remarkable issue was that one patient filled in the CF and signed it, but ticked only “no” for all obligatory and optional modules. Therefore, the patient actively refused to participate in the BFCC registry using a valid CF. However, according to the digital representation the patient had consented to all modules as if he/she had ticked only “yes.” This error in digitizing the CF belonging to the criterion “correctness” (electronic CF corresponded to paper-based origin) led to the correction of the digital representation and, consequently, to the exclusion and deletion of this patient's dataset.

Quality Issues That Were Not Resolved by the Recruiting Hospital

Of all the CF quality issues identified, two had not been addressed by the recruiting hospital at the end of baseline recruitment in February 2019: the two individual cases involved (1) a missing CF-scan (indicator “legal certainty,” category “no CF-scan uploaded “) and (2) optional module selections that deviated from the CF-scan (indicator “correctness,” category “consent of optional modules is wrongly digitized”). Both issues were processed and closed by TTP based on individual case decisions as follows.

• (1) In the case of a missing CF-scan that was not submitted or uploaded significantly past the recruitment end date, it must be assumed that there was no CF or an CF exists that does not ensure legal certainty. In such a case, the participant's data have to be completely deleted from the BFCC databases because the data cannot be processed in a legally secure manner on basis of an IC. Consequently, complete data deletion was performed by the TTP and the other BFCC data-processing agencies. Therefore, this dataset is “lost” to research.

• (2) In the case of the discrepancies in module selections between CF-scan and digital CF, the participant's will was clearly and legally validly represented on the paper-based CF, but not correctly entered in the BFCC system. Normally, the treating hospital had to correct the digital form but, in this case, it was agreed on shifting the responsibility to the TTP. Consequently, based on the paper-based consent at hand, a new digital CF was created by TTP to reflect the modules selected on paper. Thus, the participant's data were still viable for research and subsequent use.

Final Datapool After First Recruitment Period of the Baltic Fracture Competence Centre Project

In summary, 71 CFs in a total of 846 CFs had initial quality issues. Not all quality issues could be corrected by the respective hospitals. The main reason was that the patient was already discharged and could not be successfully recontacted. Hence, six CFs had to be declared invalid and the respective patients' data were not available for research. In conclusion, 840 of the 846 registered patients had valid CFs without quality issues after the end of the first recruitment period and completed QA. Remarkably, this means that 65 datasets could be retained due to QA so that 99.29% of all registered patients and their datasets are usable for research.

Discussion

The evaluation showed that quality issues arise even when using uniform paper-based CFs. Such issues range from systematic to one-time ones. However, QA proved that the initial quality of paper-based CF collection can be significantly improved by giving feedback, which led to clinical staff's learning curves, and reduced quality issues considerably. Nevertheless, the first few months of the visible learning curve in [Fig. 2] are biased, since correcting the CFs with quality issues was problematic at the beginning: missing CF-scans could be easily uploaded, but replacing already existing CF-scans can only be done by a data trustee to assure traceability and validity. As a consequence, exchanging CFs outside the data capture software in an encrypted way posed a technical barrier at first. For this purpose, a ticket system with point-to-point encryption was implemented by the TTP. However, the ticket system was first available in August 2018 and, then, enabled registry sites to send corrected CFs to the TTP. Therefore, most issues could not be corrected in the time between the first QA report in June 2018 and the availability of the ticket system in August 2018. This led to the increasing number and, then, the sudden drop of quality issues. The actual learning curve would probably be shorter after the first CF-QA-reports were sent and follow-up trainings were conducted, if a timely correction had been possible. However, timely feedback and, if necessary, follow-up training in consent collection successfully eliminated quality issues over time. Only the CFs' quality from Polish sites cannot be comprehensively assessed by central QA due to local QA measures.

It remains to be determined what time lag between CF collection and QA reports is acceptable and sustainable on a long-term basis. For example, the CF quality of new registry sites beginning recruitment should be closely monitored: an early feedback can reduce the number of quality issues and shorten the learning curve for clinical staff, counteracting error-promoting habits. However, the feedback should not be provided too often and increase the workload for clinical staff. Also, the clinicians' feedback indicated that quality checks in fast-paced clinical settings should be conducted in a timely manner so that clinicians can correct the CF forms together with their patients before discharge. This is especially crucial for cases with short periods of hospitalization and outpatient care.

Counteracting habits might also be crucial for digitizing paper-based forms: in the two cases where CFs included refusals the clinical staff digitized it as if the patients had consented to all modules. This may be caused by habit since the majority of patients (99.76%) consented to all modules. However, it shows that this manual data entry is error-prone. To avoid such mistakes, the patient should enter their consent data her-/himself. Implementing fully electronic CF and consent collection processes, for example, using tablet PCs, would empower patients to have full control over their data.

Since only the CFs in one local language showed the systematic quality issues of switched entries (signature was given in the “date of signature” field and the date was written in the “signature” field), it seemed to be a translation issue. Since the subheading was “signature” and the space between the heading and first data field was narrow, patients might be easily misled and sign the form without reading on. Especially, systematic quality issues like this might be avoidable by carefully drafting the CF and training the clinical staff. However, the one-time quality issues revealed how important QA including SDV is for all CFs because it is unlikely to discover such discrepancies with only sampled data. Additionally, even monthly feedback and follow-up training could not prevent all quality issues because the TTP received every month one to three new CFs with quality issues. However, Fox et al stated that for large-scale registries SDV of all items for all patients “[…] is impractical and beyond the financial scope […]”[11] (p.115). Furthermore, SDV alone is not enough, if the paper-based CF as source data has quality issues and needs to be corrected itself. A correction process needs to be established for source data as well.

The first site-specific QA report included the quality issue “missing signature(s)”—mostly the doctor's signature was missing. After this first report, the clinicians' feedback questioned the usability and privacy conformance of a doctor's signature. However, according to GCP, the signature of the person, who informed the patient about the research project, should be given after the patient signed the CF to make sure that the patient had the opportunity to discuss the contents and any questions regarding the CF.[7] Therefore, it remained mandatory for the BFCC registry.

Conclusions

The evaluation of paper-based CFs in the context of the BFCC fracture registry proved that quality issues arise even in using uniform paper-based CFs and that CF-QA contributes significantly to improving CF quality and, thus, the number of available datasets for research. Interestingly, manual data entry does lead to errors but not to a significant error rate in digitalizing paper-based CFs. Using the defined higher-level indicators with their specific BFCC criteria proved to be suitable for CF-QA because all quality issues could be assigned to one criterion. Consequently, the described quality indicators and criteria as well as the QA processes (e.g., reporting and correcting CFs) and the final QA checklist can be seen as the best practice approach, and researchers as well as data managers are welcome to apply and integrate those into their CF-QA-processes for all research projects.

Conflict of Interest

None declared.

Acknowledgment

Thanks to all BFCC participants, partners, and, especially, the clinical staff of participating hospitals, who recruited patients for the BFCC fracture registry.

Statement of Ethical Approval

All registry sites have received a positive vote from their local ethics committee and, thus, ethical approval.

Authors' Contributions

H.R. drafted the manuscript. H.R. and D.S. were involved in the conception and implementation of the CF-QA. A.J.R. and H.R. conducted the CF-QA including feedback to partner hospitals and provided numbers and statistics regarding the BFCC CFs. M.B. and H.R. prepared the CF content in coordination with partners, data protection officers, and ethics committees, and M.B. implemented the digital CF templates. W.H. advised the clinical implementation of the consent process. T.B. was responsible for coordinating all work packages of the University Medicine Greifswald within the BFCC project and revised the manuscript critically. All authors read and approved the final manuscript.

• References

• 1 European Parliament, Council of the European Union. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official J of the Eur Union L 119 2016; 1-88
• 2 World Medicine Association. Declaration of Helsinki. Ethical Principles for Medical Research Involving Human Subjects. Fortaleza, Brazil: WMA General Assembly; 2013
• 3 Vogele D, Schöffski O, Efinger K, Schmidt SA, Beer M, Kildal D. Analysis of documented informed consent forms for computed tomography: completeness and data quality in four clinics. Radiologe 2020; 60 (02) 162-168
• 4 BFCC project. Transnational Fracture Registry Platform. Published 2022. Accessed June 28, 2022 at: https://www.bfcc-project.eu/registry.html
• 5 Pommerening K, Drepper J, Helbing K, Ganslandt T. Leitfaden zum Datenschutz in medizinischen Forschungsprojekten - Generische Lösungen der TMF 2.0. Vol Bd. 11: Medizinisch Wissenschaftliche Verlagsgesellschaft; 2014
• 6 Bialke M, Penndorf P, Wegner T. et al. A workflow-driven approach to integrate generic software modules in a Trusted Third Party. J Transl Med 2015; 13: 176
• 7 European Medicines Agency. Guideline for good clinical practice E6(R2)-Step 5, EMA/CHMP/ICH/135/1995, Committee for Human Medicinal Products (editors), 2016
• 8 Nonnemacher M, Nasseh D, Stausberg J. Datenqualität in der medizinischen Forschung. Leitlinie zum adaptiven Management von Datenqualität in Kohortenstudien und Registern. 4 ed. Berlin: Medizinisch Wissenschaftliche Verlagsgesellschaft; 2014
• 9 Dente CJ, Ashley DW, Dunne JR. et al; GRIT Study Group. Heterogeneity in trauma registry data quality: implications for regional and national performance improvement in trauma. J Am Coll Surg 2016; 222 (03) 288-295
• 10 Altreuther M, Menyhei G. International validation of the Danish Vascular Registry Karbase: a vascunet report. Eur J Vasc Endovasc Surg 2019; 58 (04) 609-613
• 11 Fox KAA, Gersh BJ, Traore S. et al; GARFIELD-AF Investigators. Evolving quality standards for large-scale registries: the GARFIELD-AF experience. Eur Heart J Qual Care Clin Outcomes 2017; 3 (02) 114-122
• 12 Kodra Y, Weinbach J, Posada-de-la-Paz M. et al. Recommendations for improving the quality of rare disease registries. Int J Environ Res Public Health 2018; 15 (08) 1644
• 13 Rau H, Geidel L, Bialke M. et al. The generic Informed Consent Service gICS^®: implementation and benefits of a modular consent software tool to master the challenge of electronic consent management in research. J Transl Med 2020; 18 (01) 287

Publication History

Received: 29 June 2022

Accepted: 04 October 2022

Article published online:
09 January 2023