How to Design a Study and Write a Grant in Radiology?

Abstract

Radiology as a specialty has grown immensely over the last few decades. The advancements in diagnostic radiology are facilitated by state-of-the-art equipment and a revolution in computation. Parallelly, interventional radiology has benefited and interventional radiologists are at the forefront of several therapeutic procedures. One of the key factors in the growth of radiology is high-quality research. Although the formulation of research questions and hypotheses is similar, the research methodology in diagnostic radiology differs from other specialties. High-quality research requires funds. Thus, it is essential for academic radiologists pursuing research in radiology to be able to write successful grants and secure funds. In this review article, we discuss the strategies to design a study and write successful grants.

Keywords

grant - radiology - research

Introduction

Impactful research drives the growth of any medical specialty. Radiology is no exception. Several landmark trials have brought radiology to the forefront of patient management.[1] [2] [3] However, medicine is rapidly evolving. To be up-to-date with the recent advancements in medicine, academic radiologists need to step in and perform high-quality translational research. Formulating a strong research question, supported by a thorough literature review, is essential.[4] [5] Other issues like the sample size, study design, study budget, and statistical analysis are also key to a successful study.[6] High-quality research needs funds to perform tests, collect and analyze data, and disseminate results. Thus, academic radiologists should not only be able to plan a study but also secure funds to complete the studies successfully. In this review, we will discuss the steps in conducting research and writing grants in radiology.

Steps in Planning a Study

• Frame a research question: The question should be novel, relevant, and feasible.

• Conduct a thorough literature search: It is essential to review the literature (preferably systematically) to identify if other similar studies are already published and to critically analyze knowledge gaps.

• Estimate sample size: Sample size estimation is critical as this influences not only the statistical significance of the results of a study but also the feasibility and design of a study (single vs. multicenter).

• Estimate study budget: Assessing the study budget and the required infrastructure is essential to assess study feasibility and writing a grant.

• Defining the study design: Depending on the research question, one may choose the most appropriate study design.

• Including a biostatistician: This is an extremely important part of planning a study as a thorough a priori statistical plan influences the data collection.

Structuring a Relevant Research Question

One of the most common models for framing a research question is the “PICO (P: patient or problem; I: intervention; C: control or comparison; O: outcome)” framework as it captures the key elements essential for a focused research question.[4] PICO framework also helps in designing and reporting systematic reviews and is linked to the 2 × 2 result table.[7] Although PICO is directly applicable to interventional studies, it is less straightforward for diagnostic studies as the outcome could be the diagnostic accuracy or the clinically relevant outcome (e.g., mortality) after evaluating with a diagnostic test.

Characteristics of a Good Research Question (FINER)[5]

• F: Feasible

• I: Interesting

• N: Narrow in scope

• N: Novel

• E: Ethical

• R: Relevant

• A strong research project must have one primary research question and no more than three to four secondary research questions.

• A well-framed research question results in a “yes/no” answer.

Types of Studies

Depending on whether the researcher applied imaging intervention for research, the radiological studies can be classified as experimental or observation. The experimental studies are prospective and are guided by a well-planned or designed hypothesis or protocol. Hence, these studies generate high-quality evidence. On the other hand, the observational studies are retrospective. Although these are quick, these studies are prone to several biases ([Table 1]). We will be discussing some of these studies in greater detail below.

Table 1 Comparison of prospective and retrospective studies

Characteristics	Prospective	Retrospective
Design	Look forward Plan recruitment and data collection before patient outcomes are measured Protocol driven	Look backward Interventions or tests of interest are not originally applied with experimental intent
Biases	Low	High
Need for time and resources	High	Low
Quality of results	High	Low

Randomized Controlled Trials

Randomized controlled trials (RCTs) provide the highest level of research evidence. Randomized trials are by design prospective so the description “prospective randomized trial” is a misnomer.[8] The key characteristic of RCT is that the intervention (or index test) is assigned to the study population randomly. It must be borne in mind that RCT study design does not guarantee impeccable results. RCTs need to be well designed to be clinically relevant. An ideal RCT should help ascertain the diagnostic accuracy over multiple sites with representative radiologists and consecutive patients referred for testing.

Explanatory versus Pragmatic RCTs

Explanatory RCTs determine “diagnostic accuracy under ideal conditions.” On the other hand, pragmatic RCTs determine “real-world” accuracy.[9] The pragmatic RCTs are designed to assess how imaging tests impact the real-world clinical practice. The pragmatic trial results can be extrapolated to similar patients in different settings with a high confidence. Thus, pragmatic RCTs are more generalizable than explanatory RCTs.[10]

• Nonrandomized designs: These are more common in radiology. Nonrandomized experimental designs tend to measure test accuracy. RCT design is inefficient and impractical for diagnostic accuracy studies as participants receive only one test. Thus, the sample size of RCT for diagnostic accuracy studies will be enormous.[8]

• Diagnostic test accuracy (DTA) studies: DTA studies not only help assess sensitivity and specificity but also evaluate adverse effects, repeatability, interobserver agreement, imaging outcome, effect of training, and resource usage, and help develop imaging scores and technical aspects of radiological procedures. The key characteristics of a DTA are:

• A new test is evaluated against an independent standard test

• Participants receive both the new imaging test and a reference test

The best reference test is a true, independent reference standard.[11] However, this can be difficult to achieve in clinical practice. Hence, researchers often evaluate the new test against a standard practice.[12]

Reference Standard

The need for a reference standard is a critical feature of DTA studies. The main characteristics of a reference standard are:

• It should provide the true disease status perfectly.

• It should not be dependent on the imaging methods being tested.

• When the reference standard is imperfect and dependent on the same methods as the tests being investigated, the accuracy of a test can be overstated.

• Thus, it may be better to use a composite reference standard derived from several sources.

To illustrate this point, let us review a study. Gupta et al reported a prospective DTA study “deep-learning enabled ultrasound-based detection of gallbladder cancer (GBC).”[13] The authors recruited consecutive patients with nonacute gallbladder lesions and aimed to determine the accuracy of the deep learning algorithm in differentiating benign and malignant gallbladder lesions. They used histopathology or cytology as the reference standard for malignant gallbladder lesions, while they used a composite reference standard for diagnosing benign gallbladder lesions comprising histopathological examination of the cholecystectomy specimen or a 3-month follow-up imaging showing reduction or resolution of the gallbladder lesion.

Diagnostic Test Pathway

Like DTA studies, participants receive both index tests and standard tests (reference standard). The results from the index test are revealed only when clinical decisions have been made and documented based on results from the standard test alone, thus reflecting standard clinical practice.[12] This design allows the researcher to determine how the new test impacts the patients' clinical trajectory. Let us review a study to understand this type of study design. Taylor et al compared the diagnostic accuracy of whole-body magnetic resonance imaging (MRI) with standard imaging pathways for detecting metastatic disease in patients with newly diagnosed colorectal cancer.[14] The standard pathway comprised the standard staging investigations being done as a part of clinical care (e.g., positron emission tomography-computed tomography [CT]). Reference standards comprise a multidisciplinary consensus panel review.

Multireader multicase (MRMC) studies

• The key characteristics are:

  • The multiple study readers should be representative of the population of readers.

  • Readers should be from more than one institution.

  • Readers should have varying educational and experience backgrounds.

  • Multiple readers (typically > 5) who interpret images from all imaging methods being investigated.

Lalji et al compared the diagnostic performance of full-field digital mammography with that of contrast-enhanced spectral mammography (CESM) using an MRMC study design.[15] This retrospective study recruited 199 consecutive cases and 10 radiology including 4 with extensive CESM experience, 3 with no CESM experience, and 3 residents.

Feasibility Studies

These studies are performed to improve the precision of uncertain parameters. Feasibility studies are important to secure subsequent funding and allow the identification of adequate numbers of potentially eligible patients as well as the proportion willing to participate. As these studies are not powered to investigate a single outcome, these allow evaluation of a range of potential outcomes and help determine the outcome most appropriate for a larger definitive trial. Feasibility studies are mostly prospective. They could also be retrospective if the necessary data exists.

Pilot Studies

Pilot studies are well-planned studies that follow a feasibility study that suggests that a sufficient number of patients can be recruited, the outcomes can be measured precisely, and the methods are transferable to other centers. Pilot studies are prospective single-center studies that are critical before performing larger trials. Similar to feasibility studies, pilot studies also investigate multiple outcomes. Data from pilot studies is critical for securing funding for a larger study.

Cohort Studies

These studies compare subjects with and without a given exposure. Cohort studies can be prospective or retrospective. An example of a prospective cohort study is the comparison of high- and low-osmolar contrast media to identify adverse events attributable to the contrast media.[16] The study found that the low-osmolar contrast material led to a decreased rate of all adverse events. Risk ratio is utilized for comparing the groups.

Case–Control Study

In this type of study, the subjects are selected based on their outcomes. “Cases” are those in whom the outcome is evaluated and “controls” are randomly selected from the population that produced cases. Exposure is subsequently assessed for both groups. Case–control studies help study the impact of an imaging test on patient outcomes. An example of this type of study is the study of the impact of mammography screening on mortality due to breast cancer.[17] In case–control studies, the association is measured using the odds ratio as the actual proportion of subjects with the outcome in exposed and nonexposed is unknown.

Confounding

Confounding occurs in observational studies when the groups being compared differ for a factor that is associated with the outcome. For example, in a study comparing the impact of the method of drainage (transperitoneal vs. retroperitoneal) of necrotic pancreatic fluid collection on the clinical outcomes in acute necrotizing pancreatitis, it should be recognized that the clinical outcomes are not only affected by the route of drainage but also by several other factors including the presence of infected necrosis, organ failure, and the type of collection (acute necrotic collection vs. walled-off necrosis).[18] The authors, therefore, performed multiple subgroup analyses to alleviate confounding. Another method to handle confounding is propensity matching in which the cases and controls are matched for certain factors that are considered to affect the outcome of interest. In the study by Xie et al evaluating the impact of visceral adiposity on the severity of acute pancreatitis, propensity score matching was done for age, gender, and body mass index.[19]

Reporting Guidelines

The investigators need to comply with the reporting guidelines as these ensure best practices while conducting a study. The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network has a library that hosts a searchable database of reporting guidelines for various studies (e.g., CONSORT [Consolidated Standards of Reporting Trials] for RCTs, STROBE [Strengthening the Reporting of Observational Studies in Epidemiology] for observation studies, PRISMA [Preferred Reporting Items for Systematic Reviews and Meta-Analyses] for systematic reviews) and the other associated resources.[20]

How to Write a Grant Application?

Successful radiological studies may need funding. Thus, the academic radiologists must develop skills for writing a grant application. In this section, we will first highlight what reviewers are looking for in a grant application and then we will list some of the important strategies that help create a good grant application.

The Review Process: How Reviewers Score a Grant Proposal?

Below, we discuss the National Institutes of Health style of review.[21]

The overall impact score reflects the reviewers' assessment of the likelihood for the project to positively impact the research field(s) involved. This takes into account the following criteria:

• Significance

  • nature of problem addressed

  • scientific premise for the project

  • impact of project results on scientific knowledge, technical capability, and clinical practice

• Investigators

  • the strength and suitability of the principal investigator and the collaborators/coinvestigators for the proposal

  • appropriate experience and training

  • ongoing record of accomplishments

  • complementary and integrated expertise for collaborative projects

• Innovation

  • novelty, refinement, or improvement of the theoretical concepts, approaches, or methodologies, instrumentation, or interventions to the field of research

• Approach

  • feasibility

  • appropriateness and robustness of the overall strategy, methodology, and the specific aims of the project presented in an unbiased manner

  • potential problems, alternative strategies, and benchmarks for success

  • justification of inclusion/exclusion of individuals based on age, gender, and ethnicity

  • ethical aspects and protection of the participants from the risks of research

  • timeline

• Environment

  • required infrastructure, laboratory services, equipment, and staff

• Budget

  • reasonable and commensurate with the proposal

Grant Writing

The investigators should keep the following points in mind while writing a grant ([Fig. 1]):

• Title

  • The title should reflect the disease and the type of study

  • It should be concise without any complex terminology for a wider audience, for example, for a study evaluating the role of abbreviated MRI in screening of hepatocellular carcinoma (HCC), a good title may be “utility of abbreviated noncontrast MRI as a screening tool for HCC in cirrhotic patients” as it provides information regarding the patient population (cirrhotics), type of imaging test (noncontrast MRI), and the outcome of interest (screening)

• Abstract

  • The abstract should contain all the key aspects of the project, including the knowledge gap and the importance of the project in the field, specific aims, methods, and the anticipated results

  • As there is often a word limit for abstract, it is essential to be as precise as possible and avoid any form of redundancy

• Introduction

  • It forms the basis for proposing the research project

  • It should be focused to the project problem (e.g., a specific disease state) rather than starting with a general description

  • It should highlight the work already done in the field to tackle the problem and the existing knowledge gaps

  • Finally, the introduction should address how the proposal aims to fill in the knowledge gap

• Research question

  • Adopt a PICO framework to form your research question[4]

  • Refer to FINER framework for assessing the suitability of your research question[5]

  • A thorough literature search is essential to identify knowns and knowledge gaps in literature which should be clearly stated in the proposal

  • Clearly state the need for or benefit of the proposed method/investigation by highlighting its significance and how it adds to the literature

• Hypothesis

  • Hypothesis should be clear, concise with limited number of confounding factors and should be subject to empirical testing

  • Data availability/feasibility of data acquisition to substantiate the hypothesis should be clearly brought out

• Goals

  • The overarching goals of the proposal should be clearly stated

  • An ideal proposal should add to existing knowledge base and enhance professional development

• Aims and objectives

  • The specific aims should be clearly stated

  • The primary and secondary objectives should be identified

  • The objectives should be highlighted with brief methodology and expected outcome(s), for example, for a study evaluating the performance of deep learning methods for noninvasive detection of Her2Neu mutation in GBC,[21] the following aims may allow the reviewer to appropriately assess the study:

    • Aim 1: To develop a deep learning radiomics (DLR) model for identification of HER2 status and prediction of response to anti-HER2-directed therapy in unresectable GBC.

    • Hypothesis: Using a retrospective cohort with unresectable GBC (N = 150), a DLR model from CT images will have high accuracy (area under the curve [AUC] ≥ 0.90) for identifying HER2 status and high accuracy (C-statistics ≥ 0.80) for predicting best overall response to anti-HER2-directed therapy.

    • Aim 2: To prospectively validate the DLR model for identification of HER2 status and prediction of response to anti-HER2-directed therapy in unresectable GBC.

    • Hypothesis: Using a prospective cohort with unresectable GBC (N = 75), the above DLR model from CT images will have high accuracy (AUC ≥ 0.85) for identifying HER2 status and high accuracy (C-statistics ≥ 0.75) for predicting best overall response to anti-HER2-directed therapy.

• Participants/target audience(s)

  • Clearly highlight the characteristics of the group(s) to be investigated

  • State where the results will be applicable to the same group(s) or a wider group(s)

• Study design

  • Justify the choice of the research (study) design and why it is the best for your proposal

• Methodology

  • This section should be written in detail and should be reproducible

  • Describe sample size estimation ensuring that you recruit a sufficient number of subjects/patients

  • Highlight what all approval will be required or have these been already obtained

  • Discuss whether an informed consent will be needed and include a detailed informed consent form (as an appendix)

  • Explain how the described methods are appropriate to test the hypothesis

  • The radiological or ancillary tests/procedures should be described in detail

  • The result interpretation should be described

  • The outcomes should be clearly defined

  • It is useful to add a flowchart showing the study design ([Fig. 1]) (https://clinicaltrials.gov/study/NCT05716620)

• Data collection and statistical analyses

  • Who will collect the data and how (enclose a detailed patient record form)

  • The plan to assess reliability and validity of results (e.g., interreader agreement) should be clearly stated

  • Describe the statistics that you plan to use and whether this will need a biostatistician

• Outcomes and conclusions

  • State the expected outcomes

  • What are the limitations that you anticipate and are there any plans to circumvent them

• Budget justification

  • The overall project budget is guided by the specific funding guidelines

  • A funding agency may not allow certain heads, for example, travel or overheads

  • Including a detailed year-wise justification for each head (nonrecurring [equipment/software], manpower, consumables, contingency, travel) is critical

  • Provide a split up in each head (if applicable), for example, in consumables—how many tests will be done and the cost of each test in a year

  • In demanding a software or equipment, the investigator should justify such a demand as the host institute may already be having a related equipment or software. The investigator has to convince the reviewers and funding agency that such a demand is essential for successful project implementation

• Team credentials

  • This is one of the most important factors that determine the success of grant application

• References

  • All important references should be included

  • The references should be updated/latest

  • The reference style should be uniform throughout the manuscript

• Appendices

  • All pro formas including the patient information sheet, informed consent form, and supplementary information can be included here

Fig. 1 Study design template.

Tips

Fig. 2 Tips for grant writing. A successful grant application must aim to answer a novel research question with a wider public health impact. The success of funding depends on certain factors including the multidisciplinary nature of research, strong personal and team credentials demonstrating the ability to accomplish research, and availability of resources and environment conducive to research. Preliminary work related to the proposed research also strengthens the grant application.

• Pursue the research and not the grant ([Fig. 2])

• Ensure novelty and avoid duplication of previous research

• Choose a topic of wider public health importance

• Engage experts from multiple disciplines

• Have an idea of the results even before submitting the grant

  • – Have a preliminary data

• Adhere to the guidelines including page limits

• The project goals should be realistic and commensurate with the investigators' expertise, timeline, and budget

• Be organized so that the proposal is easy to follow

  • – Use sections, subsections, heading, and subheadings

  • – Use diagrams, figures, and flowcharts with detailed legends

  • – Use indents, bold, italics, and utilize white spaces effectively

• Use simple language with short sentences

• Avoid jargons

• Avoid excessive acronyms

• Use active voice as much as possible

• Review and proofread yourself for flow, language, and grammar

• Seek help from your colleagues to check for any errors

There are several useful sources that help in grant writing.[22] [23] [24]

Conflict of Interest

None declared.

• References

• 1 Pisano ED, Gatsonis C, Hendrick E. et al; Digital Mammographic Imaging Screening Trial (DMIST) Investigators Group. Diagnostic performance of digital versus film mammography for breast-cancer screening. [published correction appears in N Engl J Med. 2006 Oct 26;355(17):1840] N Engl J Med 2005; 353 (17) 1773-1783
• 2 Kim K, Kim YH, Kim SY. et al. Low-dose abdominal CT for evaluating suspected appendicitis. N Engl J Med 2012; 366 (17) 1596-1605
• 3 Weiss CR, Hafezi-Nejad N. Interventional radiology: past, present, and future. Radiology 2023; 308 (01) e230809
• 4 Huang X, Lin J, Demner-Fushman D. Evaluation of PICO as a knowledge representation for clinical questions. AMIA Annu Symp Proc 2006; 2006: 359-363
• 5 Hulley S, Cummings S, Browner W. et al. Designing Clinical Research. 3rd ed. Philadelphia, PA:: Lippincott Williams and Wilkins;; 2007
• 6 Smits M, Rockall A, Constantinescu SN, Sardanelli F, Martí-Bonmatí L. Translating radiological research into practice-from discovery to clinical impact. Insights Imaging 2024; 15 (01) 13
• 7 Luijendijk HJ. How to create PICO questions about diagnostic tests. BMJ Evid Based Med 2021; 26 (04) 155-157
• 8 Halligan S, Kenis SF, Abeyakoon O, Plumb AAO, Mallett S. How to avoid describing your radiological research study incorrectly. Eur Radiol 2020; 30 (08) 4648-4655
• 9 Bossuyt PM, Olsen M, Hyde C, Cohen JF. An analysis reveals differences between pragmatic and explanatory diagnostic accuracy studies. J Clin Epidemiol 2020; 117: 29-35
• 10 Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Threats to applicability of randomised trials: exclusions and selective participation. J Health Serv Res Policy 1999; 4 (02) 112-121
• 11 Bossuyt PM, Reitsma JB, Bruns DE. et al; STARD Group. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Radiology 2015; 277 (03) 826-832
• 12 Bossuyt PM, Irwig L, Craig J, Glasziou P. Comparative accuracy: assessing new tests against existing diagnostic pathways. [published correction appears in BMJ. 2006 Jun 10;332(7554):1368] BMJ 2006; 332 (7549) 1089-1092
• 13 Gupta P, Basu S, Rana P. et al. Deep-learning enabled ultrasound based detection of gallbladder cancer in northern India: a prospective diagnostic study. Lancet Reg Health Southeast Asia 2023; 24: 100279
• 14 Taylor SA, Mallett S, Beare S. et al; Streamline investigators. Diagnostic accuracy of whole-body MRI versus standard imaging pathways for metastatic disease in newly diagnosed colorectal cancer: the prospective Streamline C trial. Lancet Gastroenterol Hepatol 2019; 4 (07) 529-537
• 15 Lalji UC, Houben IP, Prevos R. et al. Contrast-enhanced spectral mammography in recalls from the Dutch breast cancer screening program: validation of results in a large multireader, multicase study. Eur Radiol 2016; 26 (12) 4371-4379
• 16 Bettmann MA, Heeren T, Greenfield A, Goudey C. Adverse events with radiographic contrast agents: results of the SCVIR Contrast Agent Registry. Radiology 1997; 203 (03) 611-620
• 17 Moss SM, Summerley ME, Thomas BT, Ellman R, Chamberlain JO. A case-control evaluation of the effect of breast cancer screening in the United Kingdom trial of early detection of breast cancer. J Epidemiol Community Health 1992; 46 (04) 362-364
• 18 Verma N, Maurya M, Gupta P. et al. Retroperitoneal versus transperitoneal percutaneous catheter drainage of necrotic pancreatic collections: a comparative analysis. Abdom Radiol (NY) 2022; 47 (05) 1899-1906
• 19 Xie J, Xu L, Pan Y. et al. Impact of visceral adiposity on severity of acute pancreatitis: a propensity score-matched analysis. BMC Gastroenterol 2019; 19 (01) 87
• 20 Accessed October 14, 2024 at: https://www.equator-network.org/
• 21 Accessed October 14, 2024 at: https://ichgcp.net/clinical-trials-registry/NCT05718115
• 22 Accessed October 14, 2024 at: https://www.rsna.org/research/funding-opportunities/grant-writing-resources
• 23 Accessed October 14, 2024 at: https://grants.nih.gov/
• 24 Accessed October 14, 2024 at: https://wellcome.org/grant-funding/guidance/prepare-to-apply/how-to-write-wellcome-grant-application

Publication History

Article published online:
09 January 2025