Clinical trial transparency in gastrointestinal endoscopy research

Abstract

Background Under-reporting of clinical trial results can lead to negative consequences that include inhibiting propagation of knowledge, limiting the understanding of how devices work, affecting conclusions of meta-analyses, and failing to acknowledge patient participation. Therefore clinical trial transparency, through publication of trial results on ClinicalTrials.gov or in manuscript form, is important. We aimed to examine clinical trial transparency in endoscopic clinical trials.

Methods The ClinicalTrials.gov database was searched for endoscopy trials up to October 2019. Adherence to the reporting of results to the database or in publication form was recorded for each trial.

Results The final analysis included 923 trials, of which 801 were completed and 122 were either terminated or suspended. Results were available either on ClinicalTrials.gov or in publication for 751/923 trials (81.4 %). Other fields have reported a publication rate of 40 %–63 %. Results were available on ClinicalTrials.gov for 168 trials (18.2 %) and in the form of a publication for 720 trails (78.0 %).

Conclusions Compared with other fields in medicine, endoscopy clinical trials have a high rate of clinical trial transparency. However, there is room for improvements as close to one-fifth of trials fail to report results and 81.8 % do not report results to ClinicalTrials.gov.

Introduction

ClinicalTrials.gov is a web-based database maintained by the National Institutes of Health and National Library of Medicine. It is available to anyone worldwide for the registration of a clinical trial. At inception of the database, the investigators were not required to report trial results to the website. However, this led to under-reporting of trial results. This can lead to negative consequences that include inhibiting propagation of knowledge, limiting the understanding of how devices work, affecting conclusions of meta-analyses, and failing to acknowledge patient participation in helping to advance science. In cases of therapeutic or procedural trials, not reporting negative results or adverse events may even lead to patient harm. Selective reporting of clinical trial results or failure to report adverse events can be due to the interests of the study sponsors [1]. Therefore, public disclosure of clinical trial results is critical to achieving transparency [2] and is essential for ethical medical practice and population health concerns [3]. Section 801 of the Food and Drug Administration Amendments Act (FDAAA) was created to overcome this and expanded the legal requirements for trial reporting at ClinicalTrials.gov. The FDAA requires the submission of summary results data for trials registered on ClinicalTrials.gov within 1 year of trial completion, irrespective of peer-review publication [1] [4].

Overall, the literature has shown that trials registered on ClinicalTrials.gov have an unacceptably low rate of adherence to the FDAAA requirement. DeVito et al. examined 4209 such registered trials from 2018 to 2019 [4]. The authors found that results for only 40 % of the trials were reported within the 1-year timeline and only 64 % were reported at any time. Industry sponsors were more likely to be compliant than non-industry. Other studies have shown similar results among clinical trials registered at ClinicalTrials.gov [2] [5] [6] [7] [8]. Recently our group examined the publication rates in registered gastroenterology trials [9]. Of the 2429 trials, 1824 (75.1 %, 95 %CI 73.4 %–76.8 %) had results available, but only 29 % reported results at ClinicalTrials.gov. We concluded that improvement in result reporting at ClinicalTrials.gov was needed in gastroenterology.

In the aforementioned study by our group, gastrointestinal endoscopy clinical trials were not included in the analysis. Therefore, the extent of reporting of results for gastrointestinal endoscopy clinical trials registered with ClinicalTrials.gov remains unknown. Thus, the aim of our study was to examine the reporting of results for endoscopy-based clinical trials on ClinicalTrials.gov and in full publication.

Methods

Data source

The search for trials registered on ClinicalTrials.gov was conducted on 30 September 2021. Trials were searched for any endoscopic intervention. The search was conducted by a librarian (S.L.T.) according to a pre-defined search strategy (see the online-only Supplementary material).

Inclusion and exclusion criteria

All the registered trials for adults (age > 18 years) marked as completed, terminated or suspended up to 1 October 2019 were included. We allowed for a 2-year period from the time of trial completion to allow time for reporting and publication of the results [9] [10]. Exclusion criteria can be found in the Supplementary material.

Transparency of studies and data abstraction

The type of data available in ClinicalTrials.gov can be found in the Supplementary material. Four authors (A.R., D.W., A.R., F.R., Y.R.) searched PubMed, Google Scholar, and Web of Science for publication of trial results in a peer-reviewed journal using the study title, listed investigators, clinical trial number (NCT number), trial aim, and intervention. For trials that were initially reported as abstracts, a further search was conducted to identify a peer-reviewed journal publication after the abstract publication. Registered trials for which a publication could not be found were searched by two additional authors (S.G. and A.J.T.) to ensure a true negative result.

Definitions

Definitions used in this study can be found in the Supplementary material. Terms defined included: positive trial, interventional trial, observational trial, interventions, unknown trials, funding source, clinical end points, nonclinical end points, and reasons for termination.

Statistical analysis

Availability of results (available vs. not available) was the dependent variable of interest and all the trials included in the analysis were divided into two groups (i. e. results available and results not available). Trial characteristics included as independent variables in the analysis were trial end point (met vs. not met), type of study (interventional vs. observational), study status (completed vs. terminated or suspended), type of intervention, study phase, funding source, type of study end point (clinical vs. nonclinical), number of study sites, country of origin, and median duration of trials. Trials were divided into two groups based on date of trial completion before or after 1 January 2008 to assess whether the FDAAA affected the reporting of results at ClinicalTrials.gov. We hypothesized that any of these variables could affect the trial result availability. Reason for termination was analyzed separately for terminated or suspended trials.

Descriptive statistics were used to analyze differences in trial characteristics between the two study groups. Categorical variables were described as proportions and analyzed using chi-squared test or Fisher’s exact test. Continuous variables were described as median (interquartile range [IQR]) and analyzed using Wilcoxon rank-sum test. Multivariate logistic regression model with backward selection was used to analyze the effect of various trial characteristics on result availability. Effect sizes of variables associated with trial result availability were presented as odds ratios (OR), and precision of OR measurement was assessed with 95 %CIs. Result reporting or publication rate for various endoscopic interventions were analyzed separately. A two-sided P value of < 0.05 was considered significant. The analysis was performed with SAS software, version 9.4 (SAS Institute Inc., Cary, North Carolina, USA).

Results

A total of 12 460 trials were found from the initial search, of which 4276 were duplicates. Of 8184 unique registered trials, 3434 were ongoing as of 1 October 2019. Of the remaining 4750 trials, 1031 were excluded based on the study status, 446 were excluded because they were pediatric trials, 88 were excluded because completion date was missing, and 2262 were excluded for not being related to endoscopy. The final analysis included 923 trials, of which 801 were completed and 122 were either terminated or suspended ([Fig. 1]).

Fig. 1 Flow diagram showing search results, details of included and excluded trials.

Characteristics of trials with available results

Results were available either on ClinicalTrials.gov or in the form of a publication for 751/923 trials (81.4 %, 95 %CI 78.7 %–83.8 %) ([Table 1]). Results were available on ClinicalTrials.gov for 168 trials (18.2 %) and in the form of a publication for 720 trials (78.0 %). Among the 720 publications, 82 (11.4 %) were abstracts and 638 (88.6 %) were peer-reviewed journal publications. Of the 82 abstracts, 29 (35.4 %) were later published as peer-reviewed journal articles. The primary end point was met in 588 out of 751 trials (78.3 %). Result availability on ClinicalTrials.gov was low for positive trials compared with negative trials (P < 0.001). Result availability in the form of a publication was high for positive trials compared with negative trials (P < 0.001). Interventional trials (573/685, 83.6 %) were more likely to have results available compared with observational trials (178/238, 74.8 %; P < 0.01). Device-based trials (evaluating a specific device) were more likely to be reported than procedure-based trials (evaluating a new procedure using known endoscopic tools) (P < 0.001). Results were more likely to be available for multicenter trials compared with single-center trials (P < 0.01) or trials without known status of study sites (P = 0.03). Trials registered from Europe (245/292, 83.9 %) and Asia (171/195, 87.7 %) were more likely to report results than trials registered from North America (269/352, 76.4; P = 0.01). There were no other differences noted in trial results reporting by country of trial registration. Completion date before or after 1 January 2008, funding source, type of end point, or trial duration did not affect result availability ([Table 1]). Reason for termination did not affect result availability of the terminated trials ([Table 1]).

Table 1 Characteristics of trials by result availability.

Total trials (n = 923)	Results available (n = 751)		Results not available (n = 172)	P value
Study date, n (%)
Before 1 January 2008	47 (78.3)		13 (21.7)	0.53
After 1 January 2008	704 (81.6)		159 (18.4)
Primary end point met, n (%)	Yes (n = 588)	No (n = 163)	–
On ClinicalTrials.gov				< 0.001
Yes	115 (19.6)	53 (32.5)
No	473 (80.4)	110 (67.5)
In publication				< 0.001
Yes	583 (99.2)	137 (84.0)
No	5 (0.8)	26 (16.0)
Study type, n (%)				< 0.01
Interventional	573 (83.6)		112 (16.4)
Observational	178 (74.8)		60 (25.2)
Study status, n (%)				< 0.001
Completed	689 (86.0)		112 (14.0)
Terminated or suspended	62 (50.8)		60 (49.2)
Intervention, n (%)				< 0.001
Procedure	231 (80.5)		56 (19.5)
Device	268 (87.6)		38 (12.4)
Drug	101 (82.8)		21 (17.2)
Miscellaneous	84 (83.2)		17 (16.8)
Unknown	67 (62.6)		40 (37.4)
Phase, n (%)				0.09
I	19 (66.3)		11 (36.7)
II	31 (79.5)		8 (20.5)
III	66 (83.5)		13 (16.5)
IV	53 (86.9)		8 (13.1)
Not known/not applicable	575 (81.9)		127 (18.1)
Funding source, n (%)				0.48
NIH or federal	37 (78.7)		10 (21.3)
Industry	152 (78.8)		41 (21.2)
Other	562 (82.3)		121 (17.7)
End point, n (%)				0.33
Clinical	681 (90.7)		70 (9.3)
Nonclinical	160 (93.0)		12 (7.0)
Centers, n (%)				< 0.01
Single	504 (79.4)		131 (20.6)
Multiple	245 (86.6)		38 (13.4)
Not known	2 (40.0)		3 (60.0)
Country of trial origin, n (%)				0.01
North America	269 (76.4)		83 (23.6)
Europe	245 (83.9)		47 (16.1)
Asia	171 (87.7)		24 (12.3)
Other	32 (76.2)		10 (23.8)
Multiple countries	33 (82.5)		7 (17.5)
Trial duration, median (IQR), days	760 (427–1280)		912.5 (441.5–1432.50)	0.09
Reason for termination for terminated or suspended trials (n = 122), n (%)				0.06
Enrollment issues	19 (41.3)		27 (58.7)
Safety concern, adverse events, interim analysis	13 (81.2)		3 (18.8)
Medical futility or lack of efficacy	9 (60.00)		6 (40.0)
Issues related to funding, personnel, supplies, local or federal regulation	14 (51.8)		13 (48.2)
Unclear	7 (38.9)		11 (61.1)

NIH, National Institutes of Health; IQR, interquartile range.

Result availability by type of endoscopy-based trials

Result availability by various types of endoscopy-based trials was assessed ([Table 2]) and there was no statistically significant difference between any two groups (P = 0.17).

Table 2 Result availability by specific gastrointestinal endoscopy type included in the study.

	Type of endoscopy-based trials	Result availability, n/N (%)
1.	Esophagogastroduodenoscopy	201/261 (77.0)
2.	Colonoscopy	215/255 (84.3)
3.	Small-bowel endoscopy	81/99 (81.8)
4.	Pancreaticobiliary	254/308 (82.5)

Logistic regression

Logistic regression showed that result availability was higher for completed trials than for terminated or suspended trials (OR 7.58, 95 %CI 4.86–11.83; P < 0.001) ([Table 3]). Trials with device-based interventions (OR 5.65, 95 %CI 3.18–10.02; P < 0.01), procedure-based interventions (OR 3.47, 95 %CI 2.02–5.99; P < 0.001), drug-based interventions (OR 6.95, 95 %CI 2.97–16.26; P = 0.01), and miscellaneous (OR 3.28, 95 %CI 1.62–6.68; P < 0.001) interventions were more likely to be reported than trials with unknown interventions. Phase III (OR 3.46, 95 %CI 1.19–10.05; P = 0.02) or IV (OR 3.24, 95 %CI 1.04–10.11; P = 0.02) trials and trials with phase status listed as not known or not applicable (OR 4.07, 95 %CI 1.58–10.49; P = 0.02) were more likely to have result availability than phase I trials. Multicenter trials were more likely to have result availability than trials with unknown status of study sites (OR 16.67, 95 %CI 1.30–213.49; P = 0.02).

Table 3 Logistic regression model for trial characteristics affecting trial result availability.

Trial characteristics	OR (95 %CI)	P value
Completed vs. terminated or suspended trials	7.58 (4.86–11.83)	< 0.001
Interventions
Device vs. unknown	5.65 (3.18–10.02)	< 0.01
Procedure vs. unknown	3.47 (2.02–5.99)	< 0.001
Drug vs. unknown	6.95 (2.97–16.26)	0.01
Miscellaneous vs. unknown	3.28 (1.62–6.68)	< 0.001
Phase
III vs I	3.46 (1.19–10.05)	0.02
IV vs I	3.24 (1.04–10.11)	0.02
Not known/ not applicable vs. I	4.07 (1.58–10.49)	0.02
Center
Multicenter vs. unknown study site status	16.67 (1.30–213.49)	0.02

OR, odds ratio.

Discussion

In this study, we examined the rate of result availability for gastrointestinal endoscopy-based trials registered with ClinicalTrials.gov. Transparency in endoscopy clinical trials is extremely important given the invasive nature of the trials. It is important for the community to be aware of devices that did not meet primary end points or that had unintended adverse events, so that these trials are not repeated and cause undue harm to patients. Overall we found 81.4 % of included trials had available results. Overall this rate is higher that the result availability rate for clinical trials in general gastroenterology (75 %) and other fields (ranging from 40 % to 63 %) [3] [9] [10] [11]. However close to 20 % of completed trials did not have any results published in a journal or reported on ClinicalTrials.gov, and thus there is room for improvement. We did find certain factors associated with higher result availability, namely completed trials, trials with well-defined interventions (i. e. devices, procedure or drugs), phase 3 or phase 4 trials, and multicenter trials. Of these, the associations for completed trials and interventional trials were strong and precise (OR > 3 with narrow 95 %CsI), whereas associations with trial phase and study sites were weak and imprecise (OR close to 1 and very wide 95 %CIs). The latter could be due to small sample size or chance associations.

Despite the encouraging high publication rate, only 18 % of study results were reported to the ClinicalTrials.gov website. Ideally, this should be 100 % as trials that do not report results are not compliant with federal law. It is important for the endoscopy community to have ease of access to clinical trial protocols and results in one easy-to-find location, without having to perform a separate search for a publication in a journal. In addition, ClinicalTrials.gov is freely available whereas many publications require a subscription or charge for their articles to be viewed.

Endoscopy clinical trial transparency can improve through the following mechanisms. Individuals who perform research should be aware of the FDAAA requirement and adhere to it. Institutions should monitor the ClinicalTrials.gov page for trials performed at their institutions and ensure the trials are updated. Journals could require the ClincialTrials.gov page (or equivalent trial reporting website) to be updated prior to submission of the trial manuscript. Finally, the government could implement enforcement policies that all clinical trials conducted adhere to the FDAAA regulations.

Our study does have limitations. We chose to evaluate only the ClinicalTrials.gov database to examine registered clinical trial result reporting in endoscopy. There are other databases used outside the United States that were not included in this study and such databases could be studied to assess result availability rates in other countries. However, over 48 % (448/923) of the trials included in the current study originated from outside the USA and no difference in result reporting was noted between countries on multivariate logistic regression. Despite a detailed search, it is possible that a publication could have been missed, which would introduce bias and a lower publication rate. However, this is unlikely to change the overall message given the high publication rate found. Finally, it is possible that our search missed trials focused on endoscopy.

In conclusion, we found endoscopy trials registered on ClinicalTrials.gov to have higher publication rates compared with other fields (81 %). However, there is room for improvement for publication of trials results in any form (19 % of trials), and for publication of trial results specifically to the ClinicalTrials.gov database (81.8 % of trials). With greater awareness for this requirement by researchers and enforcement by institutions, journals, and the government, higher reporting rates (and thus greater clinical trial transparency) would be expected.

Competing interests

Arvind J. Trindade is a consultant for Pentax Medical. The remaining authors declare that they have no conflict of interest.

• References

• 1 Zarin DA, Tse T, Williams RJ. et al. The ClinicalTrials.gov results database – update and key issues. N Engl J Med 2011; 364: 852-860
• 2 Saito H, Gill CJ. How frequently do the results from completed US clinical trials enter the public domain? A statistical analysis of the ClinicalTrials.gov database. PLoS One 2014; 9: 1-9
• 3 Lassman SM, Shopshear OM, Jazic I. et al. Clinical trial transparency: a reassessment of industry compliance with clinical trial registration and reporting requirements in the United States. BMJ Open 2017; 7: e015110
• 4 DeVito NJ, Bacon S, Goldacre B. Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study. Lancet 2020; 395: 361-369
• 5 Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012; 344: 1-7
• 6 Becker JE, Krumholz HM, Ben-Josef G. et al. Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA 2014; 311: 1063-1065
• 7 Ross JS, Mulvey GK, Hines EM. et al. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 2009; 6: e1000144
• 8 Ross JS, Tse T, Zarin DA. et al. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ 2012; 344: d7292
• 9 Garg S, Rizvi A, Wee D. et al. Gastroenterology clinical trials transparency: an analysis of publication rates from the ClinicalTrials.gov database. Am J Gastroenterol 2022; 117: 180-183
• 10 Psotka MA, Latta F, Cani D. et al. publication rates of heart failure clinical trials remain low. J Am Coll Cardiol 2020; 75: 3151-3161
• 11 Miller JE, Korn D, Ross JS. Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. BMJ Open 2015; 5: e009758

Publication History

Received: 17 February 2022

Accepted after revision: 18 May 2022

Accepted Manuscript online:
18 May 2022

Article published online:
20 July 2022