Keywords ophthalmology residency - San Francisco match - SF match - electronic residency application
service - ERAS - accreditation council for graduate medical education - ACGME - residency
match
Because ophthalmology has generally not included the intern year in residency training,[1 ] candidates for an ophthalmology residency position have been required to apply using
the two different residency application services: (1) the San Francisco (SF) Match
Central Application Service (CAS) for the ophthalmology residency, and (2) the Association
of American Medical College's Electronic Residency Application Service (ERAS), for
their intern year.[2 ]
[3 ] This process is not only costly but also time-consuming, as each application service
requires duplicate data entry in very different formats.
The CAS is an online residency application system established by the Association of
University Professors of Ophthalmology (AUPO) to help match PGY-2 ophthalmology positions
between applicants and ophthalmology residency programs.[2 ] Apart from the independent plastic surgery and ophthalmology residencies, all other
specialties utilize the ERAS, which was developed by the Association of American Medical
Colleges (AAMC) as a mean for applicants to electronically apply to accredited residency
positions in the United States.[3 ] For most programs, ophthalmology candidates are currently required to apply separately
for a PGY-1 internship position through ERAS.[4 ]
Both residency application systems are similar in which they both require information
regarding applicant demographics, work/volunteer experiences, research activities,
personal statement, publications, letters of recommendation, board scores, grades,
etc. However, a key difference between them is how those data are input by the applicant.
The CAS form is composed of five text boxes for information regarding “career objectives,”
“specialty elective(s) and related activities,” “honors, awards, and achievements,”
“public service and activities,” and “outside interests and hobbies.” Applicants have
the ability to format the information within each text box however they choose. In
ERAS, a standardized curriculum vitae is developed with structured data entered by
the applicant. Additionally, ERAS has the ability to upload more than three letters
of recommendation, as well as several personal statements, allowing the applicant
to customize which letters and statements are sent to specific schools.
Motivated by multiple factors including a white paper proposing an integrated intern
year that would allow for additional months of ophthalmology training,[5 ] the Accreditation Council for Graduate Medical Education (ACGME) has mandated ophthalmology
training programs integrate the internship year into one of two formats by July 2021.[4 ] In the “joint” training model, the intern year will be provided by a different department
or a different institution than the ophthalmology training. In contrast, the “integrated”
model will require all 48 months of education (i.e., PGY1–4) to be under the authority
and direction of the ophthalmology program director. Given the complexity introduced
by these changes to the matching process (some programs will require separate applications
for the intern year, some will not), we set out to identify applicant preferences
regarding the match and to evaluate applicant user experiences with both application
systems in an effort to evaluate their use in the future.
Previous studies have assessed various perspectives regarding the ophthalmology match
process from medical school, residency program, and applicant viewpoints.[6 ]
[7 ]
[8 ] In a recent study, Venincasa et al[6 ] described applicant preferences regarding the ophthalmology application process
and identified a desire among applicants for centralized scheduling of interviews
through the SF Match web site. In a survey administered to ophthalmology residency
program directors, department chairpersons, and members of residency selection committees,
Nallasamy et al[7 ] found that 66% of respondents preferred the current system with the separation of
the ophthalmology match and internship match. Although 81% of respondents were satisfied
with CAS, only 38% were happy with the subheadings and presentation of application
material. They identified a need for an interactive, online database that sorted and
grouped applicants, as well as a single sheet application with specific candidate
information. The purpose of the present study is to examine applicant experiences
with both the SF Match and the ERAS residency application systems, as well as to identify
applicant preferences for the ophthalmology match in the future.
Methods
A survey was developed to assess ophthalmology residency applicants' experience and
preference for the ophthalmology match from the 2019 and 2020 ophthalmology match
cycles. This study was deemed exempt by the Institutional Review Board (IRB00206236)
of Johns Hopkins University School of Medicine as responses to the questionnaire included
no identifiable data and were otherwise anonymous. Participation was voluntary and
by completing the survey, ophthalmology residency applicants consented to participate
in the study and share their deidentified survey responses.
Ophthalmology residency applicants for the 2019 and 2020 ophthalmology match cycles
were invited to complete an online survey regarding their experiences using both the
SF Match CAS and ERAS, as well as their preferences for the ophthalmology match going
forward. For the first phase of the study, 196 applicants from the 2018 to 2019 ophthalmology
residency application cycle were selected to complete the survey via a publicly available
e-mail list of some applicants that year. All 461 applicants to the Wilmer Eye Institute
during the 2019 to 2020 match cycle were invited to complete the survey during the
second phase of this study.
The final survey included 15 questions to gauge user experiences with both the SF
Match and ERAS systems, as well as preferences for the ophthalmology match in the
future. Two of the survey questions were designed to assess overall user experience
with both types of application services using a 5-point Likert's scale from “extremely
positive” to “extremely negative.” Eight questions assessed user experiences with
different aspects of each system, including the user interface, uploading required
documents (including grades, letters of recommendation, etc.), formatting the application,
and applying to individual schools. Responses to these questions used a 5-point Likert's
scale from “extremely easy” to “extremely difficult.” The last set of questions evaluated
system and match preferences, as well as qualitative feedback, for each application
system. The survey took approximately 10 minutes to complete. The survey questions
are included in the supplement ([Supplementary Material ]; available in the online version).
The survey was administered using the Qualtrics online survey platform after the match
was completed for the 2018 to 2019 and 2019 to 2020 ophthalmology cycles. Residency
applicants received an initial invitation via e-mail and three reminder e-mails before
the survey close date. Statistical analysis was performed using Excel for macOS version
2019 (Microsoft Corp., Redmond, WA) and R version 4.0.1 for macOS (R Foundation for
Statistical Computing, available at:
http://www.r-project.org
) with p < 0.05 considered statistically significant.
Results
Among 657 applicants solicited, 208 (32%) responded, including 67 (34%) of 196 in
2019 and 141 (31%) of 461 in 2020. In general, a majority of applicants had positive
experiences using both application systems. However, a larger proportion of applicants
rated their experience with CAS (162, 78%) as “somewhat positive” or “extremely positive,”
compared with their experience with ERAS (111, 53%; [Fig. 1A ]). Furthermore, a larger proportion of applicants rated their experience with SF
match as “extremely positive” (69, 33%), compared with ERAS (21, 10%).
Fig. 1 (A ) Applicant experience applying using the Central Application Service (CAS) and Electronic
Residency Application Service (ERAS). (B ) Applicant experiences with specific components of the CAS and ERAS.
Results of applicant experience with specific aspects of each application, including
the user interface, uploading all documents, formatting the application, and applying
to individual schools are shown in [Fig. 1B ]. Responses were rated on a scale from “extremely difficult” to “extremely easy.”
With regard to navigating the interface of each application system, 32 (15%) respondents
believed this was “extremely difficult” or “somewhat difficult” for CAS, compared
with 46 (22%) respondents for ERAS. Similarly, 13 (6%) respondents felt that uploading
all documents (grades, letters of recommendation, etc.) to CAS was difficult, compared
with 30 (14%) respondents for ERAS. Concerning applicant experiences applying to individual
schools, 16 (8%) applicants found this to be difficult for ERAS, compared with 50
(24%) applicants for CAS. However, with respect to application formatting (including
text alignment, word spacing, font styling, and structure of data), a greater proportion
of respondents found this to be difficult for CAS (99, 48%), compared with ERAS (46,
22%; p = 0.001).
When respondents were asked which of the application platform systems they preferred,
124 (60%) applicants preferred CAS, while 43 (21%) preferred ERAS (p = 0.007), and 34 (16%) rated them as equivalent. With regard to match preferences,
153 (74%) respondents stated that they would prefer ophthalmology not utilize the
National Resident Matching Program (NRMP), compared with 28 (13%) respondents who
did want to utilize the NRMP for ophthalmology (p < 0.001). More specifically, 108 (52%) applicants preferred the match application
process remain the same (i.e., CAS for ophthalmology and ERAS for the internship),
while 39 (19%) preferred ERAS for both the ophthalmology and intern year, with no
early match (p = 0.06). Although, not an option on the survey, 47 (23%) respondents wrote in an
answer for this question, indicating a preference for the CAS for both their ophthalmology
and intern year.
At the end of the survey, respondents were asked to provide feedback regarding each
system and 72 (35%) and 44 (21%) respondents provided feedback on CAS and ERAS, respectively.
Of the comments provided for CAS, (26%) of 72 comments were regarding formatting of
the CAS application, of which 23 (88%) were negative. In comparison, seven (16%) respondents
commented on application formatting in ERAS, of which five (71%) were negative. With
respect to all comments, four comments described the increasing costs associated with
applying to two application services. Additionally, four comments described the inability
to couples match as a disadvantage. Eight comments noted the redundancy in applying
to two systems as the ophthalmology and intern year match will be integrated for some
programs in the upcoming years.
Given the study was completed during two different match cycle years, Wilcoxon's rank-sum
tests were used to assess differences between respondent answers between the 2 years
([Table 1 ]). Although there were several elements of the application system in which there
was a statistically significant difference between the years, formatting the CAS was
the only component in which the median response changed from a negative to a positive
response by applicants in the 2019 compared with the 2020 match cycle.
Table 1
Differences in responses between applicants in the 2019 and 2020 ophthalmology match
cycle
Question
Category
2019
2020
p -Value
1
Overall experience CAS
4
4
0.006[a ]
2
Overall experience ERAS
4
3
0.008[a ]
3
User interface CAS
4
4
0.0008[a ]
4
User interface ERAS
4
4
0.001[a ]
5
Documents CAS
5
5
0.2
6
Documents ERAS
4
4
0.009[a ]
7
Formatting CAS
2
4
0.000007[a ]
8
Formatting ERAS
4
4
0.03[a ]
9
Applying CAS
5
5
0.9[a ]
10
Applying ERAS
4
4
0.01[a ]
Abbreviations: CAS, Central Application Service; ERAS, Electronic Residency Application
Service.
Note: Median responses to questions from the survey are on a 5-point Likert's scale,
with 1 corresponding to “extremely negative” and 5 with “extremely positive” for questions
1–2. Median responses to questions 3 to 10 are on 5-point Likert's scale, with 1 corresponding
to “extremely difficult” and 5 with “extremely easy.”
a
p -Value of 0.05 was deemed statistically significant.
Discussion
We found that while a majority of applicants preferred the SF Match CAS to the AAMC
ERAS, there were several aspects of the application process that were difficult for
applicants to navigate. In particular, formatting the CAS application was found to
be a difficult task by almost half of all respondents. This was also mentioned by
23 respondents in the free-text portion of the survey where respondents expressed
a desire for better formatting tools. More specifically, applicants cited issues with
the formatting of their final application not matching the formatting they used on
the online application form, including spacing and bold/italic text. However, compared
with the ERAS system, fewer respondents felt that navigating the user interface, uploading
documents to the system, and applying to individual schools in the SF Match application
system were difficult tasks.
Similarly, a previous study by Nallasamy et al noted that the presentation of material
was an issue for members of residency selection committees.[7 ] This highlights a need for better software development for CAS. Respondents in the
2020 match cycle had a positive response with formatting CAS compared with applicants
from the 2019 match, perhaps due to changes in the software system. Along with many
other modifications, a newly revised CAS has been released for the 2021 match cycle
in which the user interface and formatting software have been updated.[9 ] Additionally, similar to ERAS, the CAS now has the ability to customize letters
and statements for specific programs. Although these changes have been currently implemented
on the CAS web site, many of these changes are still being verified and tested, with
alterations to the software being updated based on feedback from current applicants.
Further validation testing is needed to ensure that these modifications work appropriately
and improve the application process going forward.
With regard to the ophthalmology match, over half of respondents preferred the SF
Match. Ophthalmology remains one of the only specialties, apart from urology and independent
plastic surgery, to not use the NRMP for residency matching.[10 ]
[11 ] Interestingly, a vast majority of respondents (153, 74%) preferred that the ophthalmology
match not move to the NRMP in the future. When asked about their preference for the
ophthalmology match, around half of respondents desired to keep the current approach,
utilizing the SF Match for ophthalmology and the NRMP for the internship. Intriguingly,
47 (23%) respondents provided a free-text answer, noting that utilizing SF Match for
both the ophthalmology and intern year positions was the preferred approach.
One key limitation resulting from ophthalmology not participating in the NRMP match,
is that applicants are unable to participate in the match as a couple with their significant
other. In 2020, (n = 2,448 [6%]) of applicants to the NRMP participated in the couples match.[11 ] Four respondents (2%) to the questionnaire indicated frustration regarding the inability
to participate in the couples match for ophthalmology. For these candidates, the inability
to match as a couple through NRMP may provide significant stress and financial strain,
as applicants attempt to match in the same region. This may also discourage applicants
from applying into ophthalmology altogether. Currently, ophthalmology applicants are
able to participate in the NRMP as a couple for only their PGY-1 position. However,
as programs incorporate the internship, this may soon be further complicated, and
perhaps impossible, in the future.
As ophthalmology programs transition the internship into 4-year “integrated” or “joint”
training programs,[5 ] the question arises as to the best approach for ophthalmology matching in the future.
For the “joint” training programs, the NRMP must be utilized as the intern year is
not provided by the ophthalmology program. In this scenario, given the results of
the survey, utilizing SF Match for the early ophthalmology match and NRMP for the
intern year match would be more desirable than NRMP for both the internship and ophthalmology
residency matches. In contrast, the “integrated” training program would require all
48 months of education to be under the authority and direction of the ophthalmology
program director. Although not an option on our survey, 23 respondents wrote in that
they would prefer SF Match CAS with early match to be utilized for both, suggesting
this should be considered.
Although the ACGME requires ophthalmology training programs to have an “integrated”
or “joint” preliminary program by July 1, 2021, citations will not be used for failing
to integrate before July 1, 2023.[4 ] During this transition period, applicants will be required to not only apply to
“traditional” PGY-1 preliminary and/or transitional year positions, but also a combination
of “joint” and “integrated” internship programs through ERAS.[2 ] This is further complicated by the fact that “joint” internship programs may be
at the same or a different institution than the respective ophthalmology program.
Even for “joint” internship positions at the same institution, there is no current
standardized process dictated for residency programs. Some programs require applicants
to apply directly to the PGY-1 program, while others have dedicated internship slots
for their residents, in which residents apply to a designated “ophthalmology” instance
of the program. See [Fig. 2 ] for a diagram of applicant workflow and decisions when applying under these conditions.
Fig. 2 Flow diagram for applicants applying to ophthalmology residency in the era of the
integrated internship. Note that unless applying only to “integrated” programs, residents
will have to follow multiple pathways for “joint,” “integrated,” and “traditional”
internship models. CAS, Central Application Service; ERAS, Electronic Residency Application
Service; NRMP, National Resident Matching Program; PGY = postgraduate year (1, internship;
2–4, ophthalmology residency).
Ensuring applicants understand this complex process is vital to guarantee that once
the applicant matches at an ophthalmology program, they appropriately rank the respective
internship program. One mistake could cause an applicant to not match for their preliminary
internship spot or cause them to match at a different institution. This might not
only take a spot away from another applicant but would also result in an unmatched
position at the preliminary program once the “error” is corrected.
As more and more ophthalmology residency programs integrate the intern year, regardless
of the format, the results of this study highlight the desire of applicants to keep
the SF Match CAS as the application service in the future. In the coming years, we
may start to see a shift from joint programs to fully integrated programs which could
eliminate the need for two application systems entirely if all programs adopt this
model. Given the preference for CAS that applicants expressed in our study, this could
be considered for one application process in the future. This does not address the
problem of couples matching, however. Given the fact that ophthalmology is essentially
alone with its separate matching system, it is clearly not possible to accommodate
couples who wish to remain together.[12 ] This will either deter applicants from applying to ophthalmology or introduce the
added strain from being separated during training.
In addition to the lack of a couples match, the reliance on two matching programs
results in significant costs to applicants since they have to pay for two systems.
It is estimated that it costs applicants a mean of $5,704 in 2019 for the ophthalmology
application process alone, excluding costs associated with applying and interviewing
at PGY-1 positions.[6 ] For ERAS, the application fees are tiered, requiring $99 for the first 10 programs,
$16 for each of the next 11 to 20 programs, and subsequent amounts for additional
programs applied.[3 ] Although no formal estimates for the cost of applying for a PGY-1 position exist,
we estimate that it costs applicants around $2,441.20 to $3,641.20 to apply and interview
at internship positions during the traditional application process.[2 ]
[3 ]
[6 ]
[13 ]
[14 ] Of this, $150.20 is estimated for ERAS application costs.[3 ]
[13 ] We expect this cost to further increase in the transition period as applicants may
be required to apply to multiple types of internship programs, requiring the $99 fee
to apply several times for the first 10 programs applied to in each specialty. Given
67.6% of applicants in the 2019 match cycle had to obtain additional funding to offset
costs incurred by applying,[6 ] the increasing application costs during this transition phase may deter applicants
from applying into ophthalmology altogether. The total number of internship applications
required for applicants depends on whether the affiliated internship programs require
applicants to interview with their department. This number could theoretically be
reduced to one (or possibly zero), if all programs successfully transition to either
joint or integrated programs and if internship programs do not require a separate
interview. This would allow applicants to apply via ERAS to only one program after
the early ophthalmology match, or if integrated, would possibly require no further
applications. However, unless and until this situation manifests, for internship programs
that do require an interview, the additional cost of applying may be offset by joint
and integrated internship interviews that can be grouped during ophthalmology interviews,
thereby decreasing housing and transportation costs, which was deemed by applicants
to be the biggest contributor to high application costs.[6 ]
Given ERAS is the application service for essentially all other specialties, the ability
to further update their system is supported by the fees paid by a large number of
applicants and program directors. In 2015, it was estimated that ERAS application
fees costed $72 million, accounting for 40% of the AAMC's operating revenue.[15 ] This highlights the financial investment of the AAMC in ensuring ERAS's ongoing
utilization in the match process. Similarly, given the direct relationship between
the AUPO and SF Match, there is a financial benefit (∼$1 million in the 2015 and 2016
cycles[16 ]) for the AUPO to keep the CAS as the application system for the ophthalmology match
now and in the future. Given the very limited pool of programs funding the SF Match
(two residencies and some fellowships) compared with ERAS, the resources available
for supporting the ophthalmology match and its applicants' interests are limited.
Limitations
One limitation of this study is that a nonvalidated survey was administered to a portion
of all applicants to the 2019 and 2020 ophthalmology match. Additionally, the first
phase of the study included a more select group of applicants which might have produced
additional selection bias. Furthermore, in an effort to keep responses anonymous,
applicant demographics were not obtained for this survey, limiting the ability to
understand how respondent demographics compare with the applicant population as a
whole. In an attempt to limit confounding factors, applicants were invited to participate
in the study after the match was complete, which may have resulted in recall bias.
Additionally, not all application components were evaluated, including interview scheduling.
Given the results of the study published by Venincasa et al and the need for centralized
scheduling,[6 ] more studies are needed to assess all aspects of the SF Match application and to
identify areas of both improvement and strengths.
Conclusion
The results of this study suggest that applicants prefer CAS over ERAS for residency
applications. Although respondents generally had positive experiences utilizing the
system as a whole, formatting the CAS application proved to be difficult for over
half of all respondents. As all ophthalmology programs move to include the intern
year in some way and more programs develop a truly integrated program, the need for
NRMP and ERAS will evolve, as the results of this study suggest a preference for the
SF Match for ophthalmology residency and possibly the internship match. However, mores
studies are needed to directly assess how best to accommodate the new complexities
of matching in ophthalmology.
