Keywords
ophthalmology residency - virtual residency interview - resident selection - videoconferencing interview
During the COVID-19 pandemic, the Coalition for Physician Accountability, a group of national medical education organizations, recommended that residency interviews move to an online platform for the 2020 to 2021 match cycle.[1] Some non-ophthalmology residency programs have published their virtual interview experiences. However, there are no published graduate medical education (GME) virtual interview best practices. In this paper, we evaluate the efficacy of virtual interviews at our ophthalmology residency program through applicant, faculty selection committee, and resident perspectives to improve future interviews.
Materials and Methods
To reach applicants prior to interviews, we established a resident-run Instagram account, hosted two “Open House” sessions via Zoom (San Jose, CA) videoconferencing software, and arranged 30 minute to 1 hour phone calls between residents and candidates. Our program's GME website was updated with resident-produced videos about the program and links to curriculum initiatives. If selected for interview, applicants received a hand-written note from a resident and gift bag per Association of University Professors of Ophthalmology guidelines.
Prior to interviews, the selection committee held simulations to practice videoconferencing technical components. Each of the four interview days consisted of (1) academic team introductions, (2) resident question-and-answer session, (3) two selection committee panel interviews, and one-on-one interviews with (4) the chairman and (5) program director. During each interview, the applicant was asked six standardized questions. The one-on-one interviews were unstructured. Between interviews, the applicants returned to an informal virtual “waiting room” with residents and other applicants to play group games and ask residents questions. Applicants were told that they could elect to participate or turn off their video and audio, and spent 1 to 2 hours in the virtual waiting room. Our program coordinators kept track of time and “moved” applicants between interviews.
This project was determined to be exempt as a quality improvement project by our institutional review board (IRB_00136386). After interviews, 40 interviewees, eight selection members, and seven residents were emailed an anonymous survey link regarding their experiences. Participants were informed that the survey would be anonymous, voluntary, and would not influence residency selection. We utilized RedCAP (Nashville, TN) and SurveyMonkey (San Mateo, CA) for survey distribution with a reminder email 1 week after.
Surveys consisted of multiple choice, Likert scale, and short answer questions. The 12-item applicant survey assessed the perception of resources about our program, efficacy of technical components of the virtual platform, quality of interaction with residents, ability to represent oneself, ability to assess if the program was a “good fit,” and preference of interview format. Similarly, the seven-item selection committee member and resident surveys assessed perceptions of the quality of communication, technical components, ability to evaluate applicant fit, and future interview method preference.
Quantitative data were analyzed as means, medians, percentages, and totals. Qualitative data were collected in short answer free-response format. As our sample size was limited, we did not perform statistical analysis.
Results
In total, 39 of 40 (97.5%) applicants completed the survey. Eight of eight (100%) selection committee members and seven of seven (100%) residents responded to their respective surveys. Of the surveys that were completed, 100% of the questions were completed by all.
All resources in order of most to least helpful are listed in [Table 1]. [Table 2] lists the results regarding virtual platform technical quality, communication quality, and the ability to determine if applicants would be a good fit for the program. [Table 3] lists applicant, selection committee member, and resident preference for future interview format.
Table 1
Applicant assessment of resource helpfulness
|
Applicant question: which resources were helpful in making your decision to apply to and interview at the Moran Eye Center? (Select all that apply)
|
% (n) Total 24
|
|
Resident-produced videos for applicants
|
82.1 (32)
|
|
Reputation or ranking
|
79.5 (31)
|
|
Word of mouth (mentor, faculty, family, and friends)
|
79.5 (31)
|
|
Curriculum on website
|
74.4 (29)
|
|
Moran Eye Center CORE website resources
|
59 (23)
|
|
One-on-one conversations with residents via telephone or videoconferencing
|
46.2 (18)
|
|
Gift bag with handwritten note
|
43.6 (17)
|
|
Virtual open house
|
38.5 (15)
|
|
University of Utah webpages
|
38.5 (15)
|
|
Resident Instagram account
|
35.9 (14)
|
|
GME videos
|
20.5 (8)
|
|
Other
|
7.7 (3)
|
Abbreviations: CORE, Moran's Clinical Ophthalmology Resource for Education; GME, Graduate Medical Education.
Table 2
Technical quality, communication quality, and assessment of fit
|
Technical quality: audio, video, moving from one interview to the next, and internet connection
|
Quality of communication
|
Assessment of fit: was this able to be determined?
|
|
1–5 scale,[a] mean (median)
|
1–5 scale,[a] mean (median)
|
Answered yes, % (n)
|
|
Applicants
|
4.8 (5)
|
3.9 (4)
|
92.3 (36)
|
|
Selection committee
|
4.4 (4)
|
5 (5)
|
75 (6)
|
|
Residents
|
4.1 (4)
|
1.4 (0)
|
0 (0)
|
a Scale: 5 = great/very well; 4 = good/well; 3 = neutral; 2 = poor/ly; 1 = very poor/ly.
Table 3
Preference for future interviews
|
Virtual only % (n)
|
In-person only % (n)
|
Virtual only with campus visit as separate event % (n)
|
Option of in-person or virtual interview % (n)
|
|
Applicants
|
15.4 (6)
|
38.5 (15)
|
20.5 (8)
|
25.6 (10)
|
|
Faculty
|
37.5 (3)
|
50 (4)
|
0
|
12.5 (1)
|
|
Residents
|
0
|
85.7 (6)
|
0
|
14.2 (1)
|
In general, all groups thought that the technical components of the virtual platform were good (4) or very good (5) on a 1 to 5 scale ([Table 2]). Most applicants preferred to spend their breaks talking to residents (97.2%), temporarily logging off (79.5%), and talking to other applicants (53.8%). The majority of applicants believed that they had just the right amount of time to interact with residents (84.6%). Finally, 84.6% of applicants indicated that they were able to interview at more programs this year than if interviews were in-person.
Discussion
In this study, we evaluated the efficacy of our virtual ophthalmology residency interviews from the perspectives of applicants, selection committee members, and residents. We had an excellent survey response rate. Ours is the first quality improvement project to survey all parties involved in the interview process.
As visiting medical student electives were discontinued due to the COVID-19 pandemic, the applicant recruitment process started with public relations initiatives prior to interviews. Per applicants, the most popular resources were resident-made videos about our city and residency program (82.1%), conversations with residents via telephone or videoconferencing (46.2%), and a gift bag with personalized note (43.6%). Although 38.5% of applicants found the open houses to be helpful, not all respondents may have attended. For future virtual interviews, programs can consider using these methods to reach applicants.
Selection committee members and applicants overall perceived that the platform allowed for good communication and assessment of applicant fit for the program and vice-versa ([Table 2]). Unlike selection committee members, residents perceived that the quality of the communication and ability to assess fit was poor ([Table 2]). In free response format, residents submitted suggestions to implement more personal interactions through smaller group rotating waiting rooms of one to two residents per one to two applicants or hosting a virtual meet and greet the night before the interview. Several residents indicated that they did not have the opportunity to get to know applicants personally, as there was no allotment for one-on-one conversations. This indicates that videoconferenced interviews were a good platform for structured, small group conversations, and one-on-one conversations, but were poor for unstructured and large groups. This interaction is particularly meaningful in our interview process, as residents are able to submit the names of applicants they determine would be an appropriate fit with the program. In addition, historically with our in-person interviews, our program coordinators have been able to interact with applicants and gain insight into how applicants interact when there are no residents or faculty present. This year, that interaction was limited as our coordinators were tasked with moving applicants between break out rooms.
Advantages of virtual interviews include reduced cost and time for applicants and programs. Applicants can minimize costs and time for travel and lodging.[2] For programs, virtual interviews can minimize meal cost and lost clinical or surgical time for faculty and residents. University of Arizona's ophthalmology program has been conducting optional virtual interviews for 10 years and also reported on these advantages.[3] An anesthesiology program that offered either in-person or virtual interviews reported that they ranked and subsequently matched equally from both interviewee pools.[4] An internal medicine program in 2014 utilized virtual interviews with positive applicant feedback.[5] If virtual interviews continue to be utilized, we will be able to better determine if this format leads to successful residency matches.
There are possible disadvantages to virtual interviews such as limited ability to experience the city and program in person. A urology program conducted a crossover project of interviewing the same applicant both virtually and in-person with a panel of faculty, and found advantages in time efficiency and cost with virtual interviews. However, applicants expressed a reduced ability to represent themselves over videoconferencing.[6] Applicants to a different urology program thought that most aspects of the interview except resident interactions could be replicated virtually.[7] Formatting interviews to allow for structured one-on-one or small group conversations could better suit virtual interviews. More events to connect applicants to a program and its residents could address this issue.
This match cycle presented the unique challenge of quickly converting a well-established system of in-person ophthalmology residency interviews into the virtual format. About 46.3% of all survey respondents preferred in-person interviews. The majority of applicants preferred virtual only, virtual with optional in-person campus visit, or option of either in-person or virtual interview ([Table 3]). However, a minority of applicants (38%) preferred in-person interviews only. Although the virtual interview was successful technologically, the majority of residents and half of the faculty preferred in-person interviews as well. We did not ask the various stake holders why they preferred in-person interviews. We speculate that applicants may not have had a chance to openly communicate with faculty and residents or assess the city and hospital sites for themselves. Faculty and residents may be unfamiliar with virtual interviews. Residents were not able to assess applicant fit based on large group interactions. Given these potential limitations of virtual interviews, providing applicants with the option of in-person or virtual interviews could potentially bias programs toward selecting in-person interviewees for their residencies and applicants selecting programs and geographic locations that are more familiar to them. Additionally, if in-person interviewees were more likely to match, this could limit applicants with limited financial resources who could have difficulty with travel and lodging expenses.
Our project is limited by its sample size and largely qualitative nature. In critiquing a residency program of interest, applicants may have provided favorable responses. However, they were assured that surveys were anonymous. Respondents may have held subconscious bias regarding virtual interviews due to its novelty. Offering virtual interviews as an option can allow applicants to attend interviews at a greater variety of programs. Our interview evaluation helped us to better understand how virtual interviews were perceived by all parties involved. This perspective will allow us to continue to make improvements for potential future virtual interviews.
