Keywords ophthalmology residency - San Francisco match - SF match - electronic residency application service - ERAS - accreditation council for graduate medical education - ACGME - residency match
Because ophthalmology has generally not included the intern year in residency training,[1 ] candidates for an ophthalmology residency position have been required to apply using the two different residency application services: (1) the San Francisco (SF) Match Central Application Service (CAS) for the ophthalmology residency, and (2) the Association of American Medical College's Electronic Residency Application Service (ERAS), for their intern year.[2 ]
[3 ] This process is not only costly but also time-consuming, as each application service requires duplicate data entry in very different formats.
The CAS is an online residency application system established by the Association of University Professors of Ophthalmology (AUPO) to help match PGY-2 ophthalmology positions between applicants and ophthalmology residency programs.[2 ] Apart from the independent plastic surgery and ophthalmology residencies, all other specialties utilize the ERAS, which was developed by the Association of American Medical Colleges (AAMC) as a mean for applicants to electronically apply to accredited residency positions in the United States.[3 ] For most programs, ophthalmology candidates are currently required to apply separately for a PGY-1 internship position through ERAS.[4 ]
Both residency application systems are similar in which they both require information regarding applicant demographics, work/volunteer experiences, research activities, personal statement, publications, letters of recommendation, board scores, grades, etc. However, a key difference between them is how those data are input by the applicant. The CAS form is composed of five text boxes for information regarding “career objectives,” “specialty elective(s) and related activities,” “honors, awards, and achievements,” “public service and activities,” and “outside interests and hobbies.” Applicants have the ability to format the information within each text box however they choose. In ERAS, a standardized curriculum vitae is developed with structured data entered by the applicant. Additionally, ERAS has the ability to upload more than three letters of recommendation, as well as several personal statements, allowing the applicant to customize which letters and statements are sent to specific schools.
Motivated by multiple factors including a white paper proposing an integrated intern year that would allow for additional months of ophthalmology training,[5 ] the Accreditation Council for Graduate Medical Education (ACGME) has mandated ophthalmology training programs integrate the internship year into one of two formats by July 2021.[4 ] In the “joint” training model, the intern year will be provided by a different department or a different institution than the ophthalmology training. In contrast, the “integrated” model will require all 48 months of education (i.e., PGY1–4) to be under the authority and direction of the ophthalmology program director. Given the complexity introduced by these changes to the matching process (some programs will require separate applications for the intern year, some will not), we set out to identify applicant preferences regarding the match and to evaluate applicant user experiences with both application systems in an effort to evaluate their use in the future.
Previous studies have assessed various perspectives regarding the ophthalmology match process from medical school, residency program, and applicant viewpoints.[6 ]
[7 ]
[8 ] In a recent study, Venincasa et al[6 ] described applicant preferences regarding the ophthalmology application process and identified a desire among applicants for centralized scheduling of interviews through the SF Match web site. In a survey administered to ophthalmology residency program directors, department chairpersons, and members of residency selection committees, Nallasamy et al[7 ] found that 66% of respondents preferred the current system with the separation of the ophthalmology match and internship match. Although 81% of respondents were satisfied with CAS, only 38% were happy with the subheadings and presentation of application material. They identified a need for an interactive, online database that sorted and grouped applicants, as well as a single sheet application with specific candidate information. The purpose of the present study is to examine applicant experiences with both the SF Match and the ERAS residency application systems, as well as to identify applicant preferences for the ophthalmology match in the future.
Methods
A survey was developed to assess ophthalmology residency applicants' experience and preference for the ophthalmology match from the 2019 and 2020 ophthalmology match cycles. This study was deemed exempt by the Institutional Review Board (IRB00206236) of Johns Hopkins University School of Medicine as responses to the questionnaire included no identifiable data and were otherwise anonymous. Participation was voluntary and by completing the survey, ophthalmology residency applicants consented to participate in the study and share their deidentified survey responses.
Ophthalmology residency applicants for the 2019 and 2020 ophthalmology match cycles were invited to complete an online survey regarding their experiences using both the SF Match CAS and ERAS, as well as their preferences for the ophthalmology match going forward. For the first phase of the study, 196 applicants from the 2018 to 2019 ophthalmology residency application cycle were selected to complete the survey via a publicly available e-mail list of some applicants that year. All 461 applicants to the Wilmer Eye Institute during the 2019 to 2020 match cycle were invited to complete the survey during the second phase of this study.
The final survey included 15 questions to gauge user experiences with both the SF Match and ERAS systems, as well as preferences for the ophthalmology match in the future. Two of the survey questions were designed to assess overall user experience with both types of application services using a 5-point Likert's scale from “extremely positive” to “extremely negative.” Eight questions assessed user experiences with different aspects of each system, including the user interface, uploading required documents (including grades, letters of recommendation, etc.), formatting the application, and applying to individual schools. Responses to these questions used a 5-point Likert's scale from “extremely easy” to “extremely difficult.” The last set of questions evaluated system and match preferences, as well as qualitative feedback, for each application system. The survey took approximately 10 minutes to complete. The survey questions are included in the supplement ([Supplementary Material ]; available in the online version).
The survey was administered using the Qualtrics online survey platform after the match was completed for the 2018 to 2019 and 2019 to 2020 ophthalmology cycles. Residency applicants received an initial invitation via e-mail and three reminder e-mails before the survey close date. Statistical analysis was performed using Excel for macOS version 2019 (Microsoft Corp., Redmond, WA) and R version 4.0.1 for macOS (R Foundation for Statistical Computing, available at:
http://www.r-project.org
) with p < 0.05 considered statistically significant.
Results
Among 657 applicants solicited, 208 (32%) responded, including 67 (34%) of 196 in 2019 and 141 (31%) of 461 in 2020. In general, a majority of applicants had positive experiences using both application systems. However, a larger proportion of applicants rated their experience with CAS (162, 78%) as “somewhat positive” or “extremely positive,” compared with their experience with ERAS (111, 53%; [Fig. 1A ]). Furthermore, a larger proportion of applicants rated their experience with SF match as “extremely positive” (69, 33%), compared with ERAS (21, 10%).
Fig. 1 (A ) Applicant experience applying using the Central Application Service (CAS) and Electronic Residency Application Service (ERAS). (B ) Applicant experiences with specific components of the CAS and ERAS.
Results of applicant experience with specific aspects of each application, including the user interface, uploading all documents, formatting the application, and applying to individual schools are shown in [Fig. 1B ]. Responses were rated on a scale from “extremely difficult” to “extremely easy.”
With regard to navigating the interface of each application system, 32 (15%) respondents believed this was “extremely difficult” or “somewhat difficult” for CAS, compared with 46 (22%) respondents for ERAS. Similarly, 13 (6%) respondents felt that uploading all documents (grades, letters of recommendation, etc.) to CAS was difficult, compared with 30 (14%) respondents for ERAS. Concerning applicant experiences applying to individual schools, 16 (8%) applicants found this to be difficult for ERAS, compared with 50 (24%) applicants for CAS. However, with respect to application formatting (including text alignment, word spacing, font styling, and structure of data), a greater proportion of respondents found this to be difficult for CAS (99, 48%), compared with ERAS (46, 22%; p = 0.001).
When respondents were asked which of the application platform systems they preferred, 124 (60%) applicants preferred CAS, while 43 (21%) preferred ERAS (p = 0.007), and 34 (16%) rated them as equivalent. With regard to match preferences, 153 (74%) respondents stated that they would prefer ophthalmology not utilize the National Resident Matching Program (NRMP), compared with 28 (13%) respondents who did want to utilize the NRMP for ophthalmology (p < 0.001). More specifically, 108 (52%) applicants preferred the match application process remain the same (i.e., CAS for ophthalmology and ERAS for the internship), while 39 (19%) preferred ERAS for both the ophthalmology and intern year, with no early match (p = 0.06). Although, not an option on the survey, 47 (23%) respondents wrote in an answer for this question, indicating a preference for the CAS for both their ophthalmology and intern year.
At the end of the survey, respondents were asked to provide feedback regarding each system and 72 (35%) and 44 (21%) respondents provided feedback on CAS and ERAS, respectively. Of the comments provided for CAS, (26%) of 72 comments were regarding formatting of the CAS application, of which 23 (88%) were negative. In comparison, seven (16%) respondents commented on application formatting in ERAS, of which five (71%) were negative. With respect to all comments, four comments described the increasing costs associated with applying to two application services. Additionally, four comments described the inability to couples match as a disadvantage. Eight comments noted the redundancy in applying to two systems as the ophthalmology and intern year match will be integrated for some programs in the upcoming years.
Given the study was completed during two different match cycle years, Wilcoxon's rank-sum tests were used to assess differences between respondent answers between the 2 years ([Table 1 ]). Although there were several elements of the application system in which there was a statistically significant difference between the years, formatting the CAS was the only component in which the median response changed from a negative to a positive response by applicants in the 2019 compared with the 2020 match cycle.
Table 1
Differences in responses between applicants in the 2019 and 2020 ophthalmology match cycle
Question
Category
2019
2020
p -Value
1
Overall experience CAS
4
4
0.006[a ]
2
Overall experience ERAS
4
3
0.008[a ]
3
User interface CAS
4
4
0.0008[a ]
4
User interface ERAS
4
4
0.001[a ]
5
Documents CAS
5
5
0.2
6
Documents ERAS
4
4
0.009[a ]
7
Formatting CAS
2
4
0.000007[a ]
8
Formatting ERAS
4
4
0.03[a ]
9
Applying CAS
5
5
0.9[a ]
10
Applying ERAS
4
4
0.01[a ]
Abbreviations: CAS, Central Application Service; ERAS, Electronic Residency Application Service.
Note: Median responses to questions from the survey are on a 5-point Likert's scale, with 1 corresponding to “extremely negative” and 5 with “extremely positive” for questions 1–2. Median responses to questions 3 to 10 are on 5-point Likert's scale, with 1 corresponding to “extremely difficult” and 5 with “extremely easy.”
a
p -Value of 0.05 was deemed statistically significant.
Discussion
We found that while a majority of applicants preferred the SF Match CAS to the AAMC ERAS, there were several aspects of the application process that were difficult for applicants to navigate. In particular, formatting the CAS application was found to be a difficult task by almost half of all respondents. This was also mentioned by 23 respondents in the free-text portion of the survey where respondents expressed a desire for better formatting tools. More specifically, applicants cited issues with the formatting of their final application not matching the formatting they used on the online application form, including spacing and bold/italic text. However, compared with the ERAS system, fewer respondents felt that navigating the user interface, uploading documents to the system, and applying to individual schools in the SF Match application system were difficult tasks.
Similarly, a previous study by Nallasamy et al noted that the presentation of material was an issue for members of residency selection committees.[7 ] This highlights a need for better software development for CAS. Respondents in the 2020 match cycle had a positive response with formatting CAS compared with applicants from the 2019 match, perhaps due to changes in the software system. Along with many other modifications, a newly revised CAS has been released for the 2021 match cycle in which the user interface and formatting software have been updated.[9 ] Additionally, similar to ERAS, the CAS now has the ability to customize letters and statements for specific programs. Although these changes have been currently implemented on the CAS web site, many of these changes are still being verified and tested, with alterations to the software being updated based on feedback from current applicants. Further validation testing is needed to ensure that these modifications work appropriately and improve the application process going forward.
With regard to the ophthalmology match, over half of respondents preferred the SF Match. Ophthalmology remains one of the only specialties, apart from urology and independent plastic surgery, to not use the NRMP for residency matching.[10 ]
[11 ] Interestingly, a vast majority of respondents (153, 74%) preferred that the ophthalmology match not move to the NRMP in the future. When asked about their preference for the ophthalmology match, around half of respondents desired to keep the current approach, utilizing the SF Match for ophthalmology and the NRMP for the internship. Intriguingly, 47 (23%) respondents provided a free-text answer, noting that utilizing SF Match for both the ophthalmology and intern year positions was the preferred approach.
One key limitation resulting from ophthalmology not participating in the NRMP match, is that applicants are unable to participate in the match as a couple with their significant other. In 2020, (n = 2,448 [6%]) of applicants to the NRMP participated in the couples match.[11 ] Four respondents (2%) to the questionnaire indicated frustration regarding the inability to participate in the couples match for ophthalmology. For these candidates, the inability to match as a couple through NRMP may provide significant stress and financial strain, as applicants attempt to match in the same region. This may also discourage applicants from applying into ophthalmology altogether. Currently, ophthalmology applicants are able to participate in the NRMP as a couple for only their PGY-1 position. However, as programs incorporate the internship, this may soon be further complicated, and perhaps impossible, in the future.
As ophthalmology programs transition the internship into 4-year “integrated” or “joint” training programs,[5 ] the question arises as to the best approach for ophthalmology matching in the future. For the “joint” training programs, the NRMP must be utilized as the intern year is not provided by the ophthalmology program. In this scenario, given the results of the survey, utilizing SF Match for the early ophthalmology match and NRMP for the intern year match would be more desirable than NRMP for both the internship and ophthalmology residency matches. In contrast, the “integrated” training program would require all 48 months of education to be under the authority and direction of the ophthalmology program director. Although not an option on our survey, 23 respondents wrote in that they would prefer SF Match CAS with early match to be utilized for both, suggesting this should be considered.
Although the ACGME requires ophthalmology training programs to have an “integrated” or “joint” preliminary program by July 1, 2021, citations will not be used for failing to integrate before July 1, 2023.[4 ] During this transition period, applicants will be required to not only apply to “traditional” PGY-1 preliminary and/or transitional year positions, but also a combination of “joint” and “integrated” internship programs through ERAS.[2 ] This is further complicated by the fact that “joint” internship programs may be at the same or a different institution than the respective ophthalmology program. Even for “joint” internship positions at the same institution, there is no current standardized process dictated for residency programs. Some programs require applicants to apply directly to the PGY-1 program, while others have dedicated internship slots for their residents, in which residents apply to a designated “ophthalmology” instance of the program. See [Fig. 2 ] for a diagram of applicant workflow and decisions when applying under these conditions.
Fig. 2 Flow diagram for applicants applying to ophthalmology residency in the era of the integrated internship. Note that unless applying only to “integrated” programs, residents will have to follow multiple pathways for “joint,” “integrated,” and “traditional” internship models. CAS, Central Application Service; ERAS, Electronic Residency Application Service; NRMP, National Resident Matching Program; PGY = postgraduate year (1, internship; 2–4, ophthalmology residency).
Ensuring applicants understand this complex process is vital to guarantee that once the applicant matches at an ophthalmology program, they appropriately rank the respective internship program. One mistake could cause an applicant to not match for their preliminary internship spot or cause them to match at a different institution. This might not only take a spot away from another applicant but would also result in an unmatched position at the preliminary program once the “error” is corrected.
As more and more ophthalmology residency programs integrate the intern year, regardless of the format, the results of this study highlight the desire of applicants to keep the SF Match CAS as the application service in the future. In the coming years, we may start to see a shift from joint programs to fully integrated programs which could eliminate the need for two application systems entirely if all programs adopt this model. Given the preference for CAS that applicants expressed in our study, this could be considered for one application process in the future. This does not address the problem of couples matching, however. Given the fact that ophthalmology is essentially alone with its separate matching system, it is clearly not possible to accommodate couples who wish to remain together.[12 ] This will either deter applicants from applying to ophthalmology or introduce the added strain from being separated during training.
In addition to the lack of a couples match, the reliance on two matching programs results in significant costs to applicants since they have to pay for two systems. It is estimated that it costs applicants a mean of $5,704 in 2019 for the ophthalmology application process alone, excluding costs associated with applying and interviewing at PGY-1 positions.[6 ] For ERAS, the application fees are tiered, requiring $99 for the first 10 programs, $16 for each of the next 11 to 20 programs, and subsequent amounts for additional programs applied.[3 ] Although no formal estimates for the cost of applying for a PGY-1 position exist, we estimate that it costs applicants around $2,441.20 to $3,641.20 to apply and interview at internship positions during the traditional application process.[2 ]
[3 ]
[6 ]
[13 ]
[14 ] Of this, $150.20 is estimated for ERAS application costs.[3 ]
[13 ] We expect this cost to further increase in the transition period as applicants may be required to apply to multiple types of internship programs, requiring the $99 fee to apply several times for the first 10 programs applied to in each specialty. Given 67.6% of applicants in the 2019 match cycle had to obtain additional funding to offset costs incurred by applying,[6 ] the increasing application costs during this transition phase may deter applicants from applying into ophthalmology altogether. The total number of internship applications required for applicants depends on whether the affiliated internship programs require applicants to interview with their department. This number could theoretically be reduced to one (or possibly zero), if all programs successfully transition to either joint or integrated programs and if internship programs do not require a separate interview. This would allow applicants to apply via ERAS to only one program after the early ophthalmology match, or if integrated, would possibly require no further applications. However, unless and until this situation manifests, for internship programs that do require an interview, the additional cost of applying may be offset by joint and integrated internship interviews that can be grouped during ophthalmology interviews, thereby decreasing housing and transportation costs, which was deemed by applicants to be the biggest contributor to high application costs.[6 ]
Given ERAS is the application service for essentially all other specialties, the ability to further update their system is supported by the fees paid by a large number of applicants and program directors. In 2015, it was estimated that ERAS application fees costed $72 million, accounting for 40% of the AAMC's operating revenue.[15 ] This highlights the financial investment of the AAMC in ensuring ERAS's ongoing utilization in the match process. Similarly, given the direct relationship between the AUPO and SF Match, there is a financial benefit (∼$1 million in the 2015 and 2016 cycles[16 ]) for the AUPO to keep the CAS as the application system for the ophthalmology match now and in the future. Given the very limited pool of programs funding the SF Match (two residencies and some fellowships) compared with ERAS, the resources available for supporting the ophthalmology match and its applicants' interests are limited.
Limitations
One limitation of this study is that a nonvalidated survey was administered to a portion of all applicants to the 2019 and 2020 ophthalmology match. Additionally, the first phase of the study included a more select group of applicants which might have produced additional selection bias. Furthermore, in an effort to keep responses anonymous, applicant demographics were not obtained for this survey, limiting the ability to understand how respondent demographics compare with the applicant population as a whole. In an attempt to limit confounding factors, applicants were invited to participate in the study after the match was complete, which may have resulted in recall bias. Additionally, not all application components were evaluated, including interview scheduling. Given the results of the study published by Venincasa et al and the need for centralized scheduling,[6 ] more studies are needed to assess all aspects of the SF Match application and to identify areas of both improvement and strengths.
Conclusion
The results of this study suggest that applicants prefer CAS over ERAS for residency applications. Although respondents generally had positive experiences utilizing the system as a whole, formatting the CAS application proved to be difficult for over half of all respondents. As all ophthalmology programs move to include the intern year in some way and more programs develop a truly integrated program, the need for NRMP and ERAS will evolve, as the results of this study suggest a preference for the SF Match for ophthalmology residency and possibly the internship match. However, mores studies are needed to directly assess how best to accommodate the new complexities of matching in ophthalmology.
