Clinical Research Primer
Fundamentals and Primer Overview
We divided the foundational components of the Clinical Research Primer into seven principal domains, each of which represents both a landmark in the development of a peer-reviewed study, and an associated set of skills that are critical for junior scholars to develop in a deliberate fashion ([Fig. 1]).
-
Pitching and designing clinical studies.
-
Developing a research workflow.
-
Navigating the Institutional Review Board (IRB).
-
Data collection and analysis.
-
Manuscript writing and editing.
-
Submission mechanics.
-
Tracking research projects for career development.
Pitching and Designing Clinical Studies
Although the concepts for many research studies will be originated by supervising faculty or resident mentors, asking salient and novel research questions is a critical skill and one worth developing intentionally early in training. For most students, the best point of departure is an area of interest: what case, disease, treatment, anatomic location, tool, technique, or test caught your interest? Did you encounter a discussion regarding patient management that indicated an area of controversy or lack of insight? Have you noticed patients or diseases that appear less well-served by current standard-of-care treatments than you might have anticipated? These and other practically minded questions often provide the critical germ that can evolve into a thoughtful, provocative, and ultimately insightful research study.
Once a preliminary question has been formulated in your mind, the next step is to verify that it is novel—in other words, to make sure that another investigator has not already asked and attempted to answer it. Anecdotally, Google Scholar is often the most efficient avenue for conducting such a search, given that its algorithm is keyword-driven and less dependent on Medical Subject Headings than a PubMed query. If your initial search seems promising, the next step would be to more formally structure it using the PICO(S) format (or population, intervention, comparison, outcome, studies). This standardized framework will be key in communicating your clinical research question to supervising faculty or resident mentors. Please see below for an illustrative example, color-coded to coincide with elements of the PICO(S) format.
Within the context of the endoscopic third ventriculostomy (ETV) success score, do pediatric patients who fail treatment from ETV and go on to receive a ventriculoperitoneal shunt (VPS) have a higher risk of complications than those who receive a VPS without prior ETV?
If you discover that your question has already been asked and answered, several alternative strategies may warrant consideration. One is to consider whether the question should be reframed to include an alternative perspective, such as changing the population of interest (e.g., underserved, elderly, or low-middle income country patients; individuals undergoing novel treatment combinations), or selecting a novel analytic strategy (e.g., meta-analysis; population-based study). If none of these avenues yields an interesting, novel, and practical study, you may want to abandon your question in favor of a new topic or refer the question to a faculty or resident mentor for advice. Candidly, if you think a research study will not be unique or straightforward to perform at such an early developmental phase, you will likely benefit from jettisoning the idea and moving on to fresh terrain. We place this domain at the beginning of the Primer to emphasize the importance of ensuring that your question will lead to a valuable and publishable study before meaningful time and resources are invested, and note that most successful researchers consider and abandon at least 10 candidate projects for each study they commit to completing.
A critical component of this phase is formally identifying a preferred study design. For most projects led by medical students, this will mean a case reports, literature review, systematic review, meta-analyses, or observational study (e.g., cohort, case-control, or cross-sectional study), although experimental study designs may be relevant under the right circumstances (e.g., basic/translational science, anatomy, and randomized-control trials). Perhaps the most common and successful study designs for student scholars is the combination of a case report with a literature review, in which an illustrative patient is presented as a lens for surveying the preceding publications on a relatively rare disease, unconventional clinical presentation, or uncommon treatment or complication. Whether the literature review should be a Preferred Reporting Items for Systematic reviews and Meta-Analyses-compliant systematic review or a simple search often depends on the rarity of the target population; however, when in doubt, the systematic approach lends additional robustness and rigor to the study, typically leading to a more impactful and meritorious publication. Advice from resident or faculty mentors may be of particular value in finalizing these considerations in your study design.
In the Primer model, we highlight a strategy for students to develop new questions and present them to experts for review. Candidly, although we feel strongly that this is a useful and important skill for medical trainees to develop, we also recognize that it is inherently challenging and at times nearly impossible for relatively inexperienced medical students to synthesize the clinical and scientific understanding of a topic sufficiently to formulate a meaningful research question. In a more traditional model, faculty or resident mentors furnish well-vetted research questions for students, and we endorse both approaches; however, to provide a tool that is useful to the broadest possible audience, we have written the Primer to include the process of generating study questions at the level of the medical students themselves.
Once a reliable question has been defined, and the right study design selected, the final phase of this domain is to organize your pitch into a format that will broaden the PICOS question into a strategy for data collection and analysis ([Fig. 2]). More specifically, you should compose a simple outline that describes five key components:
-
PICOS question including explicit definition of the target population, sampling strategy, and study design.
-
The independent variable of interest, with a detailed, objective definition for how it will be defined and captured.
-
The outcome of interest, also with a detailed, objective definition for how it will be defined and captured.
-
The data capture and architecture plan, ideally with an accompanying draft spreadsheet.
-
A brief summary of your rationale for how the study design will adequately and directly address the PICOS question, as well as any anticipated barriers to success.
Fig. 2 Example data collection document created with Google Sheets. Medical record numbers in column A are purely fictional.
Once you are confident in your question and plan, they should be forwarded to the supervising faculty or resident for review, feedback, and final approval before you launch into data collection and the rest of your study. Several other points warrant consideration during these preliminary phases of a new research project and may similarly benefit from faculty input. Clear communication regarding authorship and author order should be emphasized, ensuring that all contributors will have meaningful and well-defined roles, while also minimizing the risk of miscommunication or unintentional offense between colleagues who had different assumptions regarding who would be the primary author. As students develop their research profile, consideration may be given to developing a research theme, tying projects together over time as serial explorations of related clinical questions, disease processes, or other salient topics. A coherent research profile will enhance residency applications and funding proposals alike while also increasing the probability of developing the expertise that is often required to conduct truly groundbreaking work in one's scientific career.
Developing a Research Workflow
Once you have defined a clinical question and an associated study design, it is worth pausing to ensure an organized approach to how you will structure your data, files, and other study materials. Organization is the foundational principle of personal project management, the cornerstone of which is a reliable, standardized, and intuitive system for keeping many different files that will accrue during a research effort readily accessible and understandable. In general, we recommend a cloud-based tool, as these lend themselves to both the distributed nature of how your time will be parsed in medical research (e.g., you will be able to access your project from home, school, the hospital, while traveling, etc.), as well as the need for collaborative access to the materials (e.g., the [PI] and any key collaborators may also need to review data, edit text, compose figures, etc.).
A key component that must be considered when selecting appropriate workflow avenues is the security of protected health information (PHI). PHI is any information revealing current or past medical information and treatment that may be linked to a person, which is protected under the Health Insurance Portability and Accountability Act (HIPAA). Thus, when collecting patient data, safeguards must be in place in accordance with HIPAA. Each academic institution provides ample training regarding this topic during the clinical research onboarding process.
Members of our research team have experimented with a variety of platforms including Dropbox, Google Drive, and OneDrive, each of which has strengths and limitations, and all of which provide the same basic feature of a stereotyped folder architecture. In our model, each PI has a parent folder, within which project-specific folders are created; those folders in turn contain subfolders for literature, data, tables, figures, manuscript, and submission. Your institution or PI may prefer a particular platform, and the team should tailor their activities accordingly; of note, as of this writing, only OneDrive and associated Microsoft 365 products have an application programming interface (API) that is compatible with HIPAA policies at most U.S. institutions. Given the considerable patient risks and legal liabilities associated with HIPAA violations, we strongly recommend direct confirmation of all pertinent institutional policies before proceeding with storing data or other materials potentially PHI on any given platform.
In tandem with the folder architecture, new projects benefit from a dedicated forum for documentation of progress and delegation of responsibilities between contributors. As with cloud storage platforms, numerous coworking platforms such as Slack, Teams, and Notion have been used across various academic contexts, each of which has a range of overlapping benefits and vulnerabilities. As with OneDrive, Microsoft Teams has a HIPAA-compliant API, which has rendered it the preferred coworking platform for our group and many similar organizations. Within Teams, a new channel is created for every project, which will be the central point for all pertinent discussions, rather than email threads, text chains, or other mechanisms that may be less inclusive, less reliably documented, and not HIPAA compliant. This also has the marked benefit of facilitating rapid on-boarding of new project members, as the entire project dialogue will be stored and threaded in a single location.
In addition to the project-specific benefits, our research team has found that the integration of new members has been significantly improved by the development of several public channels that provide shared resources for all contributors, such as manuscript or letter templates, how-to guides for literature review or meta-analyses, lists of journals with detailed information on manuscript types accepted or impact factors (IFs), and a variety of other high-impact tools for clinical research. Please see [Table 1] for an overview of commonly used platforms in clinical research.
Table 1
Common tools for conducting clinical research
|
Category
|
Example
|
Description
|
|
Organization
|
Google Drive and Microsoft Teams (document storage infrastructure)
|
These cloud-based platforms allow for the organization and access to pertinent project documents.
|
|
Communication and task deliberation
|
Slack, Microsoft Teams (chat or forum feature), and Notion
|
These platforms allow for succinct communication and collaboration, including defining team roles and interval project updates.
|
|
Project completion
|
Google Docs, Google Slides, Microsoft Word, and PowerPoint
|
These platforms allow for the completion of crucial documents, including manuscripts, figures, and tables.
|
Navigating the Institutional Review Board
Every hospital maintains an IRB, a group of internal and external representatives who assume responsibility for oversight of all biomedical research involving human subjects. Generally, IRB approval is required for all clinical studies involving more than one human subject; thus, case reports are exempt from IRB approval, although obtaining written consent from the patient or their family is still advised whenever possible. Protocols vary by team, but where possible, the expertise of a research supervisor or coordinator to assist with managing IRB relationships is preferred. We advocate for the development of IRB templates for each broad category of research study, with the understanding that the first author for each project will take responsibility for tailoring the general form to their specific protocol prior to submission. Each institution will have particular protocols that will need to be incorporated into your team's workflow; however, emphasizing a standardized and transparent approach to IRB submission, approval, and monitoring will significantly enhance the efficiency and efficacy of your clinical scholarship.
Data Collection and Analysis
If the Methods and Results are the most important components of any manuscript, so also is the data collection and analysis the most important aspect of the execution of any clinical research project. Ideally, a thoughtful approach to the Pitching and Designing phase leaves this component of the project essentially on autopilot, an exercise in follow-through that will take time, but present few obstacles. As noted above, a draft data sheet is critical to the planning phase of your project, as this ensures that your variables are clearly defined, and ready to be captured in a format that is optimized for analysis. For most clinical analyses, categorical variables should be collapsed to binaries or dummy variables that allow complete data capture with 0/1 data; continuous variables should be captured using a common unit and the number of significant figures. If possible, you should discuss with your PI prior to data collection what statistical package they prefer, as most require that certain conventions be observed to preserve data integrity and accurate analysis. For example, R, a programming language for statistical computing and data visualization, requires that blank cells be filled with “NA” (not applicable, case sensitive), and variable names have to be lowercase and in snaketext (e.g., with underscores instead of spaces between words). Similarly, the specific variables required and their associated formatting should be tailored to the anticipated analysis—issues in this domain are particularly common with meta-analysis, where a pre-hoc plan for pooling of effect sizes should be considered mandatory to avoid calculation errors or pooling of incompatible data types during analysis (e.g., heart rate and operating room).
Manuscript Writing
Although trainees new to research often express anxiety about authoring their first manuscript, when the project has been thoughtfully organized around a practical question with an appropriate study design and analysis plan, many will be surprised at how straightforward the writing becomes. Most manuscripts follow a standardized format: Abstract, Introduction, Methods, Results, Discussion, and Conclusion. The Abstract is a brief summary of key elements in the paper, including the clinical question, study design, findings, and interpretation. Abstracts are formatted similarly to the manuscript itself, although specific requirements may vary by journal, and should be limited to 250 words in general. The Introduction provides the salient background for readers to understand why the research question is important, highlighting both the need for the present study and how the anticipated findings might impact practice, with a target length of one to three paragraphs that are tightly focused on the study question. The Methods details the study design, data collection strategy, and analysis plan including statistical testing. A rigorous Methods section allows for reproducibility, which is a critical feature assessed during peer review. The Results section provides the actual findings of your study, often including both narrative components and reference to the figures and tables that present the data in a more digestible format. The Discussion is an opportunity for the authors to interpret their data and analysis, placing the study findings within the broader context invoked by the Introduction. Depending on the scope of the study, the extent of the preceding literature, and the alignment of the findings with expectations, the length of this section varies dramatically, from a cursory review where the preceding literature is limited, to an in-depth analysis that parses a complex series of arguments. Finally, the Conclusion provides a brief, one-paragraph summary of the entire study, emphasizing the key findings as they relate to the study question.
Submission Mechanics
Many projects will undergo submission to both a national meeting and a peer-reviewed journal. Conference submissions universally require an abstract, while the opportunity to include figures or tables is highly variable. Accepted abstracts are designated for either poster or brief podium (e.g., oral) presentation; details regarding how to best present your research in either format is beyond the scope of this manuscript, but will be covered elsewhere in the Primer, and should be conducted in collaboration with your faculty and resident mentors.
Submission to peer-reviewed journals of course requires the completed manuscript with its full suite of figures, tables, and other accompanying documents. Specific requirements vary by journal, and detailed instructions can be obtained on each publication's website, but generally most manuscripts should also include a formal title page with disclosure statements, a list of abbreviations, and a cover letter to the editorial office.
Selecting a journal for submission is a nuanced decision, and input from the PI is mandatory; they will be able to assess the novelty and impact of the manuscript, which in turn will help identify an optimal target. Each publication has an IF that is calculated and reported annually by a third party and which represents an estimate of the journal's influence as a function of citation rates and the IFs of those journals.[10]
[11] By virtue of their smaller audience, neurosurgery journals tend to have impact factors <5, with higher prestige journals in the 3–6 range, and smaller journals in the 1–3 range, generally speaking. Some high-impact neurosurgery publications will be better suited to a high-impact general interest, neurology, or basic science journal; however, the vast majority of clinical papers are ideal for one of our routine specialty publications.[10]
Following submission, the manuscript may be desk rejected by the editor, sent for peer-review, or rarely accepted without a request for revisions. In the most common scenario, the reviewers will either reject the manuscript or deem it potentially worthy of publication, after which you will receive a list of comments that require detailed point-by-point changes to the manuscript and formal letter responses to the reviewers. Rejected manuscripts often receive accompanying comments, which should at least prompt discussion with the PI regarding whether they should be addressed prior to resubmission to an alternative outlet. Some journals will recommend transfer to a lower-impact or open-access journal within the same publication family; this often signals a high likelihood of acceptance to that journal, but most open-access venues require authors to pay article processing fees, which may influence the PI's decision regarding whether to accept the transfer or decline and resubmit to a separate venue.
Tracking Research Projects for Career Development
In addition to project-specific organization, developing a personal system for maintaining organization across projects and even research teams will pay significant dividends as your research prowess grows. A routine spreadsheet such as Excel or Google Sheets is adequate to maintain a list of ongoing projects, although a variety of free customizable databases such as AirTable, Microsoft Lists, or Notion provide much more robust features. Metadata for each of your projects should be tracked and periodically updated, such as coauthors, project title, PI, current status, prior/current journal submissions, and for accepted manuscripts, PubMed Identifier (PMID). Given that a research record is a core component of your academic curriculum vitae and Electronic Residency Application Service application, keeping these details organized is critical.
Discussion
We report the first component of a Clinical Research Primer for Medical Students, which outlines in detail the key elements of planning and executing basic scholarly projects for junior-level trainees. To our knowledge, no such resource has previously been published with explicit emphasis on the practical aspects of successfully developing and carrying out novel clinical projects, written at the level of a medical student with little or no preceding research experience. Although informed by our perspective as students and faculty within academic neurosurgery, these guidelines are presented in a manner that is readily generalizable to essentially any field of clinical study, and we anticipate broad utilization and impact in that regard. Significantly, although contemporary UME students often perceive a need for increasing their raw publication count, we also emphasize the utility of the Primer in empowering rigorous and high-quality research, and note that students and patients alike will benefit from focusing on a smaller number of more impactful projects.
Benefits of Undergraduate Medical Education Scholarship
The ability to read and participate in clinical scholarship has become widely recognized as a vital component of modern evidence-based medical practice, due to its significant potential to positively impact essentially all aspects of clinical care. For trainees, this constitutes a mandate to master the basic parameters of critical appraisal, which in turn is informed by a robust research profile and deep experience with clinical studies. This is perhaps the strongest argument in favor of UME developing more deliberate efforts to encourage and support student scholarship, which we anticipate that works such as the current study will helpfully inform and empower.
Exposure to scholarship has been shown to increase research engagement at numerous levels of the health care apparatus, including among students, nurses, and allied health professionals.[12] Still other efforts to create research curricula at the resident level have similarly succeeded in increasing research productivity and critical understanding of medical literature.[13] In 2006, Davis et al developed a research practicum consisting of diverse teams including faculty, fellows/residents, and medical students, each of which was tasked with designing a novel clinical protocol. They found that 50% of teams completed the study, with 47% going on to conduct subsequent research,[4] again affirming that simply providing opportunities and educational materials may dramatically increase long-term scholarly engagement among UME students and other trainees.
Considered from another perspective, one of the most modifiable contributors to low levels of academic activity among medical trainees is the lack of exposure, opportunity, and mentorship for entry-level investigators. Correspondingly, resources such as the Clinical Research Primer introduced by the current study have the potential to bridge a major gap impacting students at essentially all medical schools. Encouragingly, formal research education modules are increasingly prevalent throughout UME curricula; however, these efforts are fundamentally synergistic with the current study, and both have the opportunity to substantially lower the potential barriers to both entry and success for medical student scholarship.
In addition to the primary benefits of improving clinical knowledge and engagement with evidence-based practices, research activity has numerous important secondary benefits for medical students. Among applicants in highly competitive specialties such as neurosurgery, or those seeking a position at a particularly competitive institution in essentially any field, the change from a scored grading system to a pass-fail grading system on the USMLE Step 1 has markedly increased the emphasis on UME scholarship.[14] Within neurosurgery, the trend toward a larger number of both research experiences and accepted peer-reviewed publications among matched applicants has accelerated, increasing the urgency for students to become engaged with research early in their UME experience.[15]
[16]
The current study represents an important companion effort to the existing infrastructure for student engagement with neurosurgery, such as interest groups and mentorship programs, which have been correlated with favorable rates of students successfully matching into the field.[9] Given the competitive nature of the application process, recent developments in student outreach have become prominent. For example, Koller and peers reviewed current developments in neurosurgery outreach, in addition to the development of their own initiative, the Neurosurgery Education and Research Virtual Interest Group (NERVE). They found that most current outreach initiatives occur in webinar format, in which most participants indicated an increase in neurosurgery interest and readiness for subinternships. Moreover, they observed that 52.6% of participants in the NERVE initiative presented a poster, 47.4% submitted at least one manuscript for peer-reviewed publication, and 21.1% had a manuscript accepted or published, with the majority of students indicating an increased preparedness for subinternships and residency.[8] Perhaps most importantly, the authors emphasized the critical role of research mentorship in building successful neurosurgery career development at the UME level. Our own experiences have strongly aligned with these findings, and ultimately led to the development of the Primer as a mechanism for empowering initial engagement among medical students endeavoring to explore clinical research.
Limitations and Future Direction
Given the fundamentally subjective nature of our topic, and source materials derived from the individual experiences of a collection of faculty and students at a single academic neurosurgery department, the current study is subject to a range of potential biases. This may be offset in part by the marked diversity in training pathways and clinical practices that characterize the experiences of our faculty; notwithstanding, trainees are advised to seek local input from student and faculty mentors as they attempt to develop their research profiles and programs within their institutions. Additionally, the scope of the current work as an overview of the Clinical Research Primer does not delve deeply into critical nuances, such as how to select an ideal study design or statistical test. These and other related considerations will be directly addressed in future components of the Primer; interested students are also recommended to seek advice from a collaborator with biostatistical expertise, or to consult a reference text. Although the guidelines presented in the current work have been validated through several iterations of our own academic experiences, data are lacking in terms of generalizations beyond our institutions. Finally, although the recommendations reported here are likely to enhance UME research experiences and productivity, they rest on the assumption of local support, in particular advising from engaged faculty and resident mentors. Future components of the Primer will also address strategies for building a collaborative research forum which may provide a scaffolding where such infrastructure is lacking, and students may have success venturing to adjacent departments if academic support is tenuous in their area of desired specialization, but some degree of engaged local support is essentially mandatory for the development of a robust medical student research experience.
