Over the past 50 years, the medical field has seen a marked decline in the number of physicians pursuing careers in clinical research.[1] James Wyngaarden, MD, former director of the National Institutes of Health (NIH), first noted this decrease in the late 1970s. He called clinical investigators an endangered species in an atmosphere focused on basic science over patient-oriented research.[2] Years later, Weinreb revisited this issue, underscoring the increasing difficulties that clinician-scientists face in getting funding while maintaining clinical productivity.[3] In recent decades, attempts to address this shortage have included formalizing research training during clinical years to ensure that it remains a career-long priority.[4]
Several expert committees—including the NIH Director's Panel on Clinical Research,[5] the Institute of Medicine's Clinical Research Roundtable,[6] and the Association of American Medical Colleges' second Clinical Research Task Force (CRTFII)[7]—have generated recommendations and funding mechanisms for implementing clinical research education in the United States. Specific points highlighted by these expert panels include the need for a structured, didactic curriculum that incorporates a core knowledge base; practical clinical research experience; mentorship opportunities; protected research time for mentors; and formal recognition, such as a certificate or master's degree. All of these panels emphasize the importance of introducing medical students and physicians to clinical research as early as possible in their training.[5]
[6]
[7]
The creation of the National Eye Institute (NEI) in 1968 was a seminal event for vision research. The NEI legitimized the field of ophthalmic research, its research budget growing from $24 million in 1970 to $500 million in 2000 and to $708 million in 2017.[8] Research projects supported by the NEI address the leading causes of blindness and impaired vision in the United States. The NEI currently funds multiple awards for mentored clinician-scientists, including the Mentored Clinical Scientist Research Career Development Award (K08), the Mentored Clinical Scientist Development Program Award (K12), the Mentored Patient-Oriented Research Career Development Award (K23), and the Pathway to Independence Award (K99/R00).[9]
The NEI K30 Clinical Research Curriculum Award is now a part of the NIH Clinical and Translational Science Awards (CTSA) program, which was started in 2006 to strengthen and support clinical and translational research (CTR). A national consortium of 62 academic health centers are supported to transform and accelerate CTR. Recognizing the urgent need to increase the pipeline of CTR investigators, the CTSA has emphasized Research Education and Career Development of junior CTR investigators.[10]
[11]
While there are more than 450 ophthalmology residents in the United States graduating annually, only 39 K08 and 6 K12 awards were funded by the NEI in 2017.[12] In fact, the number of physician-scientists receiving postdoctoral research training and career development awards is at an all-time low.[13] In addition, only 2% of ophthalmologists are actively involved in research activities and this number will likely decline further in the foreseeable future.[14] Thus, physician-scientists are less likely to engage in biomedical research than in the past years. The number of physicians supported by NIH training and fellowship grants has also declined, even though the number of applications for funding has increased.[3]
[13] The dearth of clinician-scientists in ophthalmology is particularly critical to address because of the increase in eye-care demands of the aging U.S. population.
Despite available NEI funding mechanisms, few ophthalmologists become clinician-scientists due to lack of time to write clinical research grants and limited research training during medical school, residency, and fellowship programs. However, ample evidence suggests that exposing students to clinical research during medical school helps increase their interest in research careers.[13]
[15] Early commitment to the career path of clinician-scientist, especially prior to or during the resident training phase, is key to retaining the best ophthalmologists in academic careers. In addition, formal mentoring programs are effective in helping junior investigators become independent researchers.[16]
In an effort to provide a structured, didactic curriculum with practical, clinical research experience for students interested in ophthalmology early in their careers, we developed an 8-week Clinical Vision Research Training and Mentoring Program for undergraduate and medical students. Wills Eye Hospital, established in 1832 as the nation's first hospital specializing in eye care, is a global leader in ophthalmology. U.S. News & World Report currently ranks Wills Eye Hospital as one of the top two ophthalmology hospitals in the country. The clinical expertise, state-of-the-art diagnostic testing center, and advanced surgical capabilities make Wills a worldwide referral center, which annually treats more than 300,000 patients. Subspecialty services include glaucoma, neuroophthalmology, retina, ocular oncology, pediatrics and ocular genetics, ocular pathology, cornea, and oculoplastic and orbital surgery. Wills also serves as the Department of Ophthalmology for Sidney Kimmel Medical College at Thomas Jefferson University and their medical students rotate through Wills during required and elective ophthalmology courses as well as clinical clerkships. This article describes the program's aims, course content, lecture series, and evaluation data during the summer of 2014 and 2015.
Methods
Setting
Program Goals and Objectives
The program was developed, implemented, and evaluated by the Director of the Glaucoma Research Center at Wills Eye Hospital. Two cohorts of students took part in the Clinical Vision Research Training and Mentoring during the summers of 2014 and 2015. The full-time, 8-week program offered clinical research training, hands-on research experience, and mentorship to 34 students from 12 institutions in Pennsylvania, New York, New Jersey, Ohio, Georgia, and the District of Columbia ([Table 1]). With the largest clinical volume of patients with eye disease in the United States, Wills offers a unique opportunity for students to gain useful clinical research experience in ophthalmology.
Table 1
Students' university enrollment (n = 34)
|
Type of student
|
Quantity
|
University represented
|
|
Undergraduate (pre-medicine)
|
4
|
Columbia University
University of Pennsylvania
Emory University
LaSalle College
|
|
Undergraduate (public health)
|
1
|
Temple University
|
|
Graduate (medical school)
|
29
|
Sidney Kimmel Medical College at Thomas Jefferson University
Drexel University College of Medicine
Temple University School of Medicine
Philadelphia College of Osteopathic Medicine
Robert Wood Johnson School of Medicine
George Washington School of Medicine
Northeast Ohio Medical University
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with permission, 2017).
The program goals and objectives were to engage students in the following:
-
Actively participate in a vision research project.
-
Conduct vision-related literature reviews.
-
Write a scientific article for a peer-reviewed journal.
-
Receive mentoring from faculty and research staff.
This program gave students a unique introduction to ophthalmology, experience in clinical research, and the skills necessary to conduct vision research. Students worked with a multidisciplinary group of research coordinators and managers, biostatisticians, data managers, and ophthalmologists to advance their knowledge, improve their manuscript writing skills, and engender positive attitudes about vision research. Mentorship was a key element of the program, with each student assigned to an investigator and a member of the research team.
Recruitment and Admissions
The 2014 and 2015 Clinical Vision Research Training and Mentoring Program was directed by the Department of Research at Wills Eye Hospital. The program was posted on the Department of Research, Wills Eye Hospital, and Sidney Kimmel Medical College Web sites. Candidates submitted a curriculum vitae, a letter of recommendation, and a cover letter to the Department of Research Director in January 2014 and January 2015. The director, faculty members, research managers, and human resources director conducted personal interviews, selected students who had an interest in ophthalmology, prior research experience, and planned to apply to medical school if they were an undergraduate student. Seventeen undergraduate and medical students were selected and enrolled each year from a total of 25 applicants in 2014 and 31 applicants in 2015.
Lecture Series
Many students entering the program had only a generalized knowledge of ocular disease and basic research methodology. To familiarize students with common ophthalmic conditions, as well as the basic principles of clinical research, an 11-hour lecture series was developed and presented by Wills Eye Hospital and Sidney Kimmel Medical College faculty. Lectures discussed glaucoma, optic neuropathy, and other ocular diseases; several ongoing research studies being conducted at Wills and the Institutional Review Board (IRB) policies ([Table 2]). Students participated in research projects related to these lectures.
Table 2
Lecture topics
|
Lecture
|
Topic
|
|
1
|
Introduction to ophthalmology
|
|
2
|
Introduction to glaucoma
|
|
3
|
Introduction to neuro-ophthalmology and optic neuropathy
|
|
4
|
Introduction to diabetes and diabetic retinopathy
|
|
5
|
Introduction to age-related macular degeneration
|
|
6
|
Introduction to cataract and refractive surgery
|
|
7
|
Introduction to smoking and eye disease
|
|
8
|
Glaucoma community outreach demonstration project
|
|
9
|
Quality-of-life study for glaucoma
|
|
10
|
Transcorneal electrical stimulation study for optic neuropathy
|
|
11
|
Institutional Review Board policies and procedures
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with permission, 2017).
Clinical Research Skills and Experience
The program provided students with a framework for research design, critical analysis, and the practical skills needed to pursue an academic career in ophthalmology. Prior to working on any research projects, students completed Collaborative Institutional Training Initiative (CITI) and electronic medical record (EMR) training.[17] The research staff created a manual and trained the students on the Wills' EMR system. After demonstrating proficiency, students utilized the EMR to recruit patients for active clinical trials and extract data for IRB-approved retrospective studies requiring ocular chart reviews. Students completed CITI training, which taught them how to identify, recruit, enroll, and retain human subjects in research studies. The CITI training also familiarized students with the Health Insurance Portability and Accountability Act (HIPAA) and its protection of patients' privacy. Students had the opportunity to obtain informed consent from patients during which they explained patients' rights as well as researchers' expectations in a clinical trial.
We also designed a 10-hour manuscript writing workshop series for enrolled students ([Table 3]). Training modules taught students how to conduct a literature review and use Endnote X5 software (Thomson Reuters, New York, NY). Students were assigned to an IRB-approved research project that was near completion and ready for the development of a manuscript. The students were allotted at least 6 to 8 hours per week for manuscript writing. Every 2 weeks, students delivered a newly written section of their manuscript, beginning with the introduction and continuing with the methods, results, and discussion sections. Several research assistants with writing experience served as peer editors and provided feedback at each juncture. The program director, research managers, and biostatistician offered ongoing feedback and reviewed the final manuscripts.
Table 3
Manuscript writing workshop and training schedule
|
Workshop
|
Topic
|
|
1
|
Electronic medical record training
|
|
2
|
Patient recruitment and data collection training
|
|
3
|
Conducting a literature review and EndNote training
|
|
4
|
Writing the introduction
|
|
5
|
Writing the research methods
|
|
6
|
Writing the data analysis
|
|
7
|
Writing the results
|
|
8
|
Writing the discussion
|
|
9
|
Developing an abstract and poster presentation
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with permission, 2017).
During the last week of the program, students identified an appropriate peer-reviewed journal for manuscript submission (based on its impact factor and subject matter) and formatted their paper accordingly. With supervision from their mentor, who was usually identified as the corresponding author, they submitted manuscripts. Students also learned how to prepare an abstract for an ophthalmology-related conference, such as the Association for Research in Vision and Ophthalmology (ARVO), as well as how to develop a research poster. If a student's abstract was accepted for a conference, he or she had the opportunity to present the research findings and were often funded by the University or a conference research travel grant.
Enrolled students gained hands-on experience by having direct exposure to IRB-approved research studies. They played an active role in subject recruitment, consenting, data collection, and data analysis with our biostatistician. Students gained experience administering baseline research questionnaires and performed visual acuity testing, visual field testing, visual evoked potential testing, electroretinographies, and contrast sensitivity testing. They also participated in weekly, department-specific research meetings with their mentors, research coordinators, and managers. During these meetings, the research team offered project updates, discussed new studies relevant to the group's major research topics, and assigned readings to clarify the background and purpose of the clinical research studies underway ([Table 4]).
Table 4
Students' responsibilities
|
• Recruit patients for research studies
|
|
• Perform electronic medical records review
|
|
• Develop a mentoring relationship with a member of the Wills Eye research staff
|
|
• Work closely with mentors and trainers to complete assignments
|
|
• Attend lectures and workshops
|
|
• Review scientific literature to develop a manuscript
|
|
• Write a manuscript about the assigned research study
|
|
• Identify peer-reviewed journals for manuscript submission
|
|
• Format final manuscript for submission to a peer-reviewed journal
|
|
• Complete deadlines for assignments related to the manuscript
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with permission, 2017).
Mentorship and Core Attitudes
Mentorship was also a significant component of the clinical research program and a strong influence on students' attitudes toward vision research.[16] Each student was teamed up with a research mentor, with whom they met with several times during the summer. This relationship offered frequent contact, performance feedback, insight into diagnostic techniques and interpretation, and, in many cases, the potential for continuing research after the summer. Many students who engaged in clinical research at Wills Eye return independently during the academic year to follow up on projects. Often, these students are able to complete research manuscripts from the data they collected and receive publication credit for their contributions.
Students were encouraged to view their research projects in the context of patient experience. Therefore, clinical and surgical shadowing opportunities were offered. The degree of clinical participation varied by specialty, and students observed their mentor in the subspecialty clinic, operating room, or the Wills Eye Emergency Room. They learned the meaning of the data variables collected and analyzed, and consequently learned how to associate these values with patients' ocular conditions and treatments. Students had the opportunity to obtain informed consent from patients during which they explained patients' rights as well as researchers' expectations in a clinical trial. It is our hope that regular involvement with patients will reinforce the translational aspects of research and keep students mindful of the importance that patient-centered research can have on vision care.
Pre- and Posttest Statistical Analysis
A pre- and posttest was administered to all students on the first and last days of the 8-week program. These tests consisted of 40 multiple choice questions within 11 topic areas and 3 to 4 questions were asked about each topic ([Table 2]). The test consisted of paired questions, so that the two tests were similar in difficulty and subject matter. A paired-sample t-test was conducted to compare scores using SAS 9.4 software (Statistical Analysis System [SAS] Institute, Cary, NC). This test was used to determine whether the mean difference between the two sets of observation is zero. The null cutoff p-value for detecting statistical significance was ≤0.05.
Satisfaction Survey
The students completed an anonymous 10-question survey of their satisfaction with the overall program, including lecture series, manuscript writing workshop series, pre- and posttests, and mentor/trainer experience. Possible responses were on a 5-point Likert scale, with answers varying from strongly disagree to strongly agree. Means and standard deviations were assessed for each question.
Results
Evaluation
Students scored statistically significantly higher on the posttest (mean [M] = 79.78%, standard deviation [SD] = 6.25) than on the pretest (M = 71.43%, SD = 8.43); p< 0.001. The largest improvement in scores was seen on the glaucoma, community outreach for glaucoma, vision and quality of life, transcorneal electrical stimulation, and IRB/consenting test sections (p < 0.001; [Table 5]).
Table 5
Pre- and posttest results: percentage of correct items (n = 34)
|
Covariate
|
Statistics
|
Pretest
n = 34
|
Posttest
n = 34
|
Parametric p-value[a]
|
|
General ophthalmology
|
n
|
34
|
34
|
0.239
|
|
Mean (SD)
|
84.71 (19.73)
|
79.41 (18.08)
|
|
Median
|
90
|
80
|
|
Min, Max
|
40, 100
|
40, 100
|
|
Glaucoma
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
52.94 (28.23)
|
84.71 (18.46)
|
|
Median
|
60
|
80
|
|
Min, Max
|
0, 100
|
40, 100
|
|
Child vision screening
|
n
|
34
|
34
|
0.021
|
|
Mean (SD)
|
78.68 (23.14)
|
88.97 (15.31)
|
|
Median
|
75
|
100
|
|
Min, Max
|
0, 100
|
50, 100
|
|
Community outreach for glaucoma
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
71.57 (32.96)
|
95.1 (11.98)
|
|
Median
|
83.33
|
100
|
|
Min, Max
|
0, 100
|
66.67, 100
|
|
Transcorneal electrical stimulation
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
52.21 (22.5)
|
78.68 (21.44)
|
|
Median
|
50
|
75
|
|
Min, Max
|
0, 75
|
25, 100
|
|
IRB/Consenting
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
64.71 (25.87)
|
88.24 (19.9)
|
|
Median
|
66.67
|
100
|
|
Min, Max
|
0, 100
|
33.33, 100
|
|
Statistics
|
n
|
34
|
34
|
0.148
|
|
Mean (SD)
|
69.61 (28.86)
|
58.82 (27.29)
|
|
Median
|
66.67
|
66.67
|
|
Min, Max
|
0, 100
|
0, 100
|
|
Diabetes and diabetic retinopathy
|
n
|
34
|
34
|
0.513
|
|
Mean (SD)
|
80 (17.75)
|
82.35 (14.58)
|
|
Median
|
80
|
80
|
|
Min, Max
|
20, 100
|
60, 100
|
|
Electronic medical records
|
n
|
34
|
34
|
0.074
|
|
Mean (SD)
|
92.35 (23.1)
|
79.41 (33.21)
|
|
Median
|
100
|
100
|
|
Min, Max
|
0, 100
|
0, 100
|
|
Smoking cessation
|
n
|
34
|
34
|
0.228
|
|
Mean (SD)
|
62.75 (17.91)
|
69.61 (22.27)
|
|
Median
|
66.67
|
66.67
|
|
Min, Max
|
0, 100
|
33.33, 100
|
|
Quality of life/SPARCS
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
86.76 (19.69)
|
68.38 (21.59)
|
|
Median
|
100
|
75
|
|
Min, Max
|
25, 100
|
25, 100
|
|
Overall score
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
71.43 (8.43)
|
79.78 (6.25)
|
|
Median
|
74.5
|
80
|
|
Min, Max
|
52.5, 82.5
|
64, 90
|
Abbreviations: IRB, Institutional Review Board; Max, maximum; Min, minimum; SD, standard deviation; SPARCS, Spaeth/Richman Contrast Sensitivity test.
a The parametric p-value is calculated by a paired t-test. Bold values indicate statistically significant.
Students strongly agreed that they were satisfied with the overall program and that the lecture series enhanced their learning (M = 4.6, using 5-point Likert scale). The lecture series (M = 4.6), the manuscript writing workshop (M = 4.6), and the editorial mechanisms for completing their manuscripts (M = 4.5) all received very favorable reviews ([Table 6]). We received several personal “thank you” notes via email and handwritten notes and have provided an example from a medical student:
Table 6
Program evaluations (n = 34)
|
Evaluation questions
|
M (SD)
|
|
I was satisfied with the program
|
4.6 (0.51)
|
|
Program goals and objectives were clear
|
4.3 (0.63)
|
|
Program was organized in a manner consistent with its stated goals
|
4.3 (0.68)
|
|
Pre- and posttest questions were fair
|
4.2 (0.82)
|
|
Adequate review mechanisms were provided for manuscript development
|
4.5 (0.56)
|
|
The course materials were valuable for learning
|
4.4 (0.73)
|
|
The lecture series and pre-residency lectures enhanced my learning
|
4.6 (0.60)
|
|
I was satisfied with the lecture series overall
|
4.3 (0.71)
|
|
The manuscript workshop series enhanced my learning
|
4.0 (0.80)
|
|
I was satisfied with the manuscript workshop series overall
|
4.6 (0.88)
|
|
The abstract and poster development sessions enhanced my learning
|
4.0 (0.79)
|
|
I was satisfied with the editorial/feedback process to prepare the manuscript
|
4.4 (0.67)
|
|
I was satisfied with the program supervision
|
4.5 (0.61)
|
Abbreviations: M, mean; SD, standard deviation.
Note: Scores range from 1 to 5 (1 = strongly disagree, 5 = strongly agree)
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with permission, 2017).
Thank you for the opportunity to participate in the Vision Research Training and Mentoring Program this summer. It was a tremendous privilege to be a part of the wonderful research community at Wills Eye Hospital and especially in the Department of Research. I am grateful to have been a part of a program where students are given such trust and independence in their work. I treasure my experience this summer and look forward to continuing work at Wills in the future.
– Daniel Kim, MS2 at Sidney Kimmel Medical College
Discussion
The Clinical Vision Research Training and Mentoring Program provided an evidence-based foundation in research methods and manuscript development for students pursuing degrees in medicine and public health. Students emerge with knowledge and skills to conduct vision research and understand basic ophthalmology. They also benefit from the support of a mentor in guiding their future career ambitions. The key strength of the Wills' approach is the vast amount of clinical vision research exposure that students gain over the course of the 8-week program. Students participate in all levels of research, from recruiting patients and gathering and analyzing data to preparing and submitting a manuscript. We hypothesize that this clinical research exposure enhanced students' core knowledge of ophthalmology and vision research skills.
Students in medical schools with traditional rigorous curricula often prefer to undertake research during the summer. Harvard University's Summer Program in Clinical Effectiveness fosters students' interest in research and provides tools for careers in academic medicine by teaching clinical research design and methodology.[18] Studies at the University of Tennessee and Vanderbilt University have examined the impact of student research fellowship programs over 25 years.[15] These programs offered mentored summer research experiences for first- and second-year medical students. A significant percentage of students performed further research, with one-third to one-half of graduates going on to pursue careers in academic medicine.[15] However, these summer research programs tend to be available only between the first and second years of medical school due to clinical rotation schedules.
The Wills Eye Program was more flexible and served not only medical students between their first and second years, but also has been abbreviated to a 1-month vision research elective format—without the lecture series and workshops—for medical students during their third or fourth year. Though created for medical students interested in applying to an ophthalmology residency, the program is easily adaptable for other specialties and may encourage research in any medical specialty. Students who decided to pursue another specialty after taking the program were able to apply basic research skills to any clinical research project.
Given that medical education programs leading to the MD/DO degree in the United States and Canada are accredited by the Liaison Committee on Medical Education (LCME), it is important to consider the recent update in LCME research requirements.[19] The LCME has created standards outlined in Functions and Structure of a Medical School, which medical schools must meet to achieve and maintain accreditation.[19] According to Standards IS-13 and IS-14, an institution that offers a medical education program should make available sufficient opportunities for medical students to participate in research and other scholarly activities of its faculty and encourage and support medical student participation.[19] Medical schools are required to describe the opportunities available for medical student participation in research, including the time periods when students may do so, the average number of students in the base year who were involved in each type of program, and the funding sources that are available to support student participation. The Wills Vision Research Training and Mentoring Program serves as a model for medical schools to successfully meet these LCME standards in the departments of ophthalmology, as well as other clinical and surgical departments.
Study Limitations
In structuring the 8-week program, we had to sacrifice some of the depth and continuity of a typical clinical-research program. Since a research project can span months or years from concept to completion, 2 months may not allow students enough time to participate in the entire process. To address this limitation, we incorporated lectures and workshops that examined various research projects at different stages. Of the students enrolled in the 2014 program, six students chose to continue working with their research mentors and several undergraduate students returned the following summer prior to beginning medical school or after their first year of medical school. As this robust program was only recently established, no longitudinal data are yet available.
