Paving the Path to Value: The Role of Registry Studies in Interventional Radiology

Abstract

The growth rate of interventional radiology (IR) procedures and practitioners has exceeded the pace of evidence development to support evidence-based practice. In the innovative and highly adaptive field of IR, there exists a tremendous need for interventional radiologists to practice evidence-based medicine as a way to maintain and improve quality of health care. Randomized controlled trials (RCTs) are considered to be the gold standard of research, providing level I evidence, but in procedural subspecialties, they remain difficult to design and implement due to challenges in randomization, blinding, and inadequate sample size. To build the foundation of evidence in IR, registry studies can play a complementary role to RCTs. Clinical data registries may offer a more practical approach to gathering outcomes data, important in this era with the advent of the Medicare Access and CHIP Reauthorization Act (MACRA), in comparison with RCTs. Properly designed registries can store an abundance of data with which high-quality observational studies can be performed. Although considered level II evidence, these registry studies will allow the evaluation of both performance and value of IR procedures, particularly in circumstances in which an RCT would not be feasible. This manuscript aims to serve as a guide for developing and participating in IR registry studies.

Keywords

evidence-based medicine - clinical data registry - randomized controlled trial

Introduction

Interventional radiology (IR) is a fast-growing specialty formed from and driven by innovation. Although the footprint of minimally invasive image-guided procedures in medicine is expanding, the evidence to support IR procedures for diagnosis and treatment of a wide range of conditions is often lacking. For IR to evolve with changing health care paradigms, more emphasis will be required at all organizational levels (societal, institutional, departmental, and individual) to assess and align with evidence- and value-based practices. At the very least, familiarity with research study design and the hierarchies of evidence will be critical for current practitioners and trainees alike.

Primary Clinical Studies and Evidence-Based Practice

Randomized controlled trials (RCTs) are the criterion standard providing level I evidence. However, RCTs necessitate exclusion criteria that can limit generalizability and may be difficult to conduct for surgical interventions due to ethical and methodologic concerns.[1] In the fast-paced world of IR, RCTs can be impractical to implement in a timely manner with sufficient quality. Between 1995 and 2014, only 117 RCTs were published in the field of IR; of those, 52.2% were found to be low quality.[2] In these situations, observational studies, in particular registry studies, may serve an important and complementary role in addressing many of the clinical questions present in IR. In addition, by building a comprehensive database of clinical outcomes through a registry, generalizability can be extended to the target population at large by including a wider range of age, ethnicity, and comorbidities.

Observational studies, usually considered level II evidence, are characterized by the lack of a direct intervention. Instead, the relationship between an exposure and disease variable is assessed in a retrospective or prospective manner. However, limitations exist in these studies, primarily centered on the presence of bias and poor internal validity due to the inability to control for confounding factors. Self-reported patient data, selection of enrolled patient population, loss to follow-up, and limited data in retrospective studies can all introduce bias.

A registry database is an organized system that uses observational methods to collect uniform data for evaluating specified outcomes for a population. Registries can include product registries, health services registries, and disease registries. Registry studies can serve many purposes, particularly when describing the natural history of disease, to measure or monitor safety and harm and to determine clinical and cost effectiveness of health care services.[3] In addition, observational studies provide a broader viewpoint with greater external validity when determining clinical applicability to the general population.

Recent analyses show comparable results between RCTs and observational studies.[4] [5] This manuscript aims to serve as a guide for developing registries and acquiring data that may ultimately change the clinical management of patients. Well-designed registry studies can provide important, valid data for the IR community at a time when the need for evidence-based practice is high. Recognizing the function and use of observational studies is the first step in the critical appraisal and design of registry studies.

Registry Design

Registry studies are of particular benefit in situations with ethical considerations, in rare diseases, and in comparative effectiveness analyses. Within IR, current registries include the University of California Invasive Placenta (UC-IP) registry, which aims to determine the outcomes of various adjuvant IR interventions (internal iliac artery occlusion balloons, aortic occlusion balloons, internal iliac artery ligation, and uterine artery embolization) for patients with invasive placenta, the Comparing Options for Management: Patient-centered Results for Uterine Fibroids (COMPARE-UF) registry for uterine fibroids and alternative treatments to hysterectomy, and the Ablation of Renal Masses Outcomes Registry (ARMOR) for tumor biology and outcome determinants in renal masses treated with percutaneous ablation. The UC-IP registry demonstrates the strength of a registry design by allowing analysis of a previously understudied rare pathology and has recently shown that aortic and internal iliac artery balloon occlusion results in significantly less intraoperative estimated blood loss compared with internal iliac artery ligation and supportive care.[6] Other situations well suited for registries include natural history studies, measures of compliance, studies of heterogeneous patient populations, and evaluations of standard medical practice.[3]

Registry Setup

In designing a registry, identification of the lead site should occur first. The principal investigator (PI) at this site will be responsible for obtaining the main study institutional review board (IRB) approval (a common IRB should be obtained to limit duration of the approval process), collecting data, performing the analysis, and securing funding if needed. Identification of collaborating sites should seek a mix of practice environments if generalizability is a goal as setting can introduce significant bias. Each site must have a site PI and collaboration with key referring physicians (including surgery, pathology, and radiology), as well as research coordinators. Research coordinators will require funding; however, trainee involvement can offset much of the workload.

High-volume sites provide many advantages. The UC MAP IR registry includes multiple sites where interventions are vastly different in contrast to ARMOR in which all sites perform the same intervention.

Identification of the inclusion principles should be clearly defined. Selection of subjects should take into account both the patient and the setting. Specific diagnostic criteria, disease subtype, and stage of disease should be specified as necessary, allowing for future subgroup analysis. For example, within the UC-IP Registry, subgroup analysis was possible due to inclusion of pathologic diagnosis and detailed operative course including adjunctive procedures. Because registry studies’ strength lies in their external validity, it is beneficial to select the study population that closely represents the target population or the population to which the results can be applied.

A collaborative effort between all site PIs should be made to identify all outcome variables prior to the initiation of data collection. This limits missing data and interpretation bias, although a balance must be struck between too little and too much data. Variables defined should include demographics and background information, diagnostic and therapeutic interventions, and outcome measures. Outcome measures, in particular, should reflect the current literature to allow for comparison of results between other clinical trials and registries.

The Society of Interventional Radiology Standardized Reporting Initiative assists in quality data collection for use in IR registries. Standardized reporting templates, which cover the majority of IR treatments, are available for download.[7]

Certain institutions may require a contract that data acquired be used in a collaborative fashion. This can delay the process of data acquisition. A Data Access and Publication Policy will need to be implemented, with details on the Data Access and Publications Committee, services available, publication guidelines, and future industry sponsorship.

A data safety monitoring board should be put in place for prospective studies. This independent committee will monitor patient safety and treatment efficacy data and intervene in the cases of overwhelming benefit or significant adverse events.

Funding Opportunities

Funding remains an important aspect of most clinical studies. Funding may be obtained from sources such as the National Institute of Health (NIH), foundations, internal institutional funds, or medical device company sponsorships. For outcomes registries, the Patient-Centered Outcomes Research Institute (PCORI) also offers funding opportunities.

Data Transfer and Interpretation

Data transfer is complicated by the health insurance portability and accountability act (HIPPA) compliance and size of data. Simple data such as laboratory values can be transferred between sites easily and with minimal cost. The sharing of imaging data can be costly but should be arranged with an institution's information technology (IT) department. Interpretation should be performed at one of two sites: either the lead study site alone or at a completely separate site. Such models limit interobserver variability in data interpretation.

Manuscript Publication

The Data Access and Publication Policy should be determined prior to initiation of the registry. Authorship of any manuscripts should be discussed prior to initiation of data analysis, along with discussion of potential subgroup analyses and interested PIs. Authorship discussion should include not only first author, senior author, and writing of the manuscript, but also co-authors and their roles in preparing the manuscript.

Limitations

Selection bias, information bias, and confounding should be acknowledged and minimized.[8] Although selection bias is, to some extent, unavoidable and is present in all clinical studies, including clinical trials, a national or international registry with multiple sites in diverse regions can limit selection bias. The use of objective outcomes, such as laboratory results, can limit information bias. Missing data can also pose a problem and have been a drawback to prior registry studies, including a study based on the National Surgical Quality Improvement Program. Importantly, confounding factors must be carefully considered during the registry design process. One important source of bias is confounding by indication: patients in a registry are not randomized to treatment and may receive a specific treatment because they are sicker. An example of indication bias leading to invalid results can be seen in a study utilizing the United States Cystic Fibrosis Foundation National Patient Registry.[9]

Discussion

The Institute for Healthcare Improvement (IHI) has developed a framework to optimize modern health care systems that considers three dimensions. Referred to as the “IHI Triple Aim Framework,”[10] the goal is to simultaneously improve the patient experience, improve the health of populations, and reduce the per capita cost of health care. Ideally, progress toward the triple-aim would occur in the context of learning health systems[11] that rapidly disseminate, integrate, and translate new knowledge into practice.

Within IR, priorities and activities should be reconsidered. Although at present the nature of a more optimal U.S. health system is a major debate, there is no doubt that the demonstration of value will be essential to success individually, and as a specialty. The role of registries moving forward will be to enable seamless collection of high-quality data to evaluate performance and value, serving a complementary role alongside RCTs. An additional benefit of participation in registry studies is the ability to compare outcomes with regional and national benchmarks, allowing for more robust quality control and improved patient care.[12] [13] IR as a primary medical specialty is now in a better position than ever to lead evidence-based clinical research, as evidenced by the implementation of clinical studies such as the PRESERVE (Predicting the Safety and Effectiveness of Inferior Vena Cava Filters) trial (NCT02381509), a joint collaboration between the Society of Interventional Radiology (SIR) and the Society for Vascular Surgery (SVS). These studies provide important information regarding the safety and efficacy of both developing and established treatment options and are invaluable for the practicing IR physician in the today's fast-paced world of medical technology and research.

Conclusion

Registry studies can be an important tool for the interventional radiologist in both clinical practice and research. It can be challenging to pursue well designed and adequately powered prospective RCTs, but registries can have a complementary role and serve as an opportunity to address knowledge gaps when other methodologies are impractical.

Registries focus on generalizability to a broad population, including a wider range of age, ethnicity, and comorbidities. Common conditions and uncommon conditions alike are amenable to registry studies. In addition, the importance of IR registries is not only for individual disease processes. With the advent of the Medicare Access and CHIP Reauthorization Act (MACRA), more data about our individual interventions is needed to compare with other specialties treating similar diseases. Through strong evidence-based clinical practice, IR will continue to grow as a primary specialty and demonstrate its value in patient care.

This way, registry studies can be more relevant in deciding clinical treatment for many population groups excluded from clinical trials. They offer more external validity and a way to perform prospective or retrospective, observational studies in circumstances in which an RCT may be unethical or otherwise unfeasible.

Conflict of Interest

None.

• References

• 1 Unger CA, Barber MD. Studying surgical innovations: challenges of the randomized controlled trial. J Minim Invasive Gynecol 2015; 22 (04) 573-582
• 2 Hong SJ, Yoon DY, Cho YK. et al. Characteristics and quality of radiologic randomized controlled trials: a bibliometric analysis between 1995 and 2014. AJR Am J Roentgenol 2016; 206 (05) 917-923
• 3 Registries for Evaluating Patient Outcomes. A User's Guide. 3rd ed.. Rockville, MD: Agency for Healthcare Research and Quality (US); 2014
• 4 Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med 2000; 342 (25) 1878-1886
• 5 Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev 2014; 4: MR000034
• 6 Ballah D, Cochran R, Dong P. et al. Outcomes of the University of California Invasive Placenta Registry: how different adjunctive procedures reduce intraoperative blood loss during cesarean hysterectomy. J Vasc Interv Radiol 2018; 29 (04) S130
• 7 Standardized Reporting. https://www.sirweb.org/practice-resources/quality-improvement2/standardized-reporting/
• 8 Psoter KJ, Rosenfeld M. Opportunities and pitfalls of registry data for clinical research. Paediatr Respir Rev 2013; 14 (03) 141-145
• 9 Rothman KJ, Wentworth III CE. Mortality of cystic fibrosis patients treated with tobramycin solution for inhalation. Epidemiology 2003; 14 (01) 55-59
• 10 Triple Aim Framework IHI. www.ihi.org
• 11 Friedman CP, Wong AK, Blumenthal D. Achieving a nationwide learning health system. Sci Transl Med 2010; 2 (57) 57cm29
• 12 Uberoi R, Das N, Moss J, Robertson I. British Society of Interventional Radiology: Biliary Drainage and Stenting Registry (BDSR) Cardiovasc. Intervent Radiol 2012; 35 (01) 127-138
• 13 Dykes TM, Bhargavan-Chatfield M, Dyer RB. Intravenous contrast extravasation during CT: a national data registry and practice quality improvement initiative. J Am Coll Radiol 2015; 12 (02) 183-191