Key Word Use in Letters of Recommendation for Ophthalmology Residency Applicants According to Race, Gender, and Achievements

Sahil Aggarwal; Seanna Grob; Dhruba Banerjee; Preston J. Putzel; Jeremiah Tao

doi:10.1055/s-0038-1675842

Journal of Academic Ophthalmology, Table of Contents

CC BY-NC-ND 4.0 · Journal of Academic Ophthalmology 2018; 10(01): e163-e171
DOI: 10.1055/s-0038-1675842

Research Article

Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

Key Word Use in Letters of Recommendation for Ophthalmology Residency Applicants According to Race, Gender, and Achievements

Sahil Aggarwal

¹School of Medicine, University of California, Irvine, California

,

Seanna Grob

²Division of Oculofacial Plastic and Orbital Surgery, Department of Ophthalmology, Gavin Herbert Eye Institute, University of California, Irvine, California

,

Dhruba Banerjee

³Department of Neurobiology and Behavior, University of California, Irvine, California

,

Preston J. Putzel

⁴Donald Bren School of Information and Computer Sciences, University of California, Irvine, California

,

Jeremiah Tao

²Division of Oculofacial Plastic and Orbital Surgery, Department of Ophthalmology, Gavin Herbert Eye Institute, University of California, Irvine, California

› Author Affiliations

Abstract

Objectives To identify differences in letters of recommendation (LORs) of applicants to a single ophthalmology residency program by gender, race, academic performance, and match outcome.

Design This was a retrospective analysis of LORS for 2,523 applicants (7,569 letters) to the University of California, Irvine ophthalmology residency program from 2011 to 2018.

Methods Programming scripts were employed to determine the number of times 22 key words from four thematic categories (standout words, ability, grindstone, and compassion) appeared in LORs for each applicant. A chi-square test was performed to assess for possible differences in the presence of each key word by the following characteristics: gender, underrepresented minority (URM) status, Alpha Omega Alpha (AOA) membership, the United States Medical Licensing Exam (USMLE) Step 1 score, and match outcome. Linear regressions were created to determine the frequency at which words in each thematic category appeared according to the same baseline characteristics.

Results In the LORs, females were more likely to be described as “empathetic” (p = 0.002), URMs were more likely to be described as “caring” (p = 0.002), high Step 1 scorers (≥240) were more likely to be described as “outstanding” (p = 0.002), and matched students were more likely to be described as “exceptional” (p = 0.001), “outstanding” (p < 0.001), and “superb” (p = 0.001). Standout words appeared more often in the LORs of AOA members, matched candidates, and high Step 1 scorers (p < 0.001 for all comparisons). “Competent” appeared more commonly in LORs for low Step 1 scorers (p < 0.001) and unmatched applicants (p = 0.001).

Conclusion This study identifies differences in LORs by gender, URM status, and achievement including successful ophthalmology residency match. Females and URMs were more likely to be described as “empathetic” and “caring,” respectively; otherwise, we detected no gender or racial disparities in key word use in LORs. Candidates with high USMLE Step 1 scores or AOA membership had a higher frequency of standout words in their LORs. Whether they were truly more qualified in various dimensions or if they benefited from a halo effect bias warrants further investigation. There was a significant difference in the number of standout words in LORs between matched and unmatched applicants, suggesting that key word frequency may be a relevant metric for LOR appraisal.

Keywords

letters of recommendation - ophthalmology - bias - gender - underrepresented minority - Alpha Omega Alpha

Full Text

References

References
1 SF Match. Ophthalmology Residency Match Summary Report 2018. Available at: https://www.sfmatch.org/PDFFilesDisplay/Ophthalmology_Residency_Stats_2018.pdf . Accessed June 15, 2018
2 Nallasamy S, Uhler T, Nallasamy N, Tapino PJ, Volpe NJ. Ophthalmology resident selection: current trends in selection criteria and improving the process. Ophthalmology 2010; 117 (05) 1041-1047
3 Grubbs Jr JR, Mian SI. Advising students interested in ophthalmology: a summary of the evidence. Ophthalmology 2016; 123 (07) 1406-1410
4 Loh AR, Joseph D, Keenan JD, Lietman TM, Naseri A. Predictors of matching in an ophthalmology residency program. Ophthalmology 2013; 120 (04) 865-870
5 SF Match. Ophthalmology Residency Standardized Letter of Reference. Available at: https://www.sfmatch.org/PDFFilesDisplay/OPHTHR_StandardizedLOR.pdf . Accessed June 16, 2018
6 Lee AG, Golnik KC, Oetting TA. , et al. Re-engineering the resident applicant selection process in ophthalmology: a literature review and recommendations for improvement. Surv Ophthalmol 2008; 53 (02) 164-176
7 Pololi LH, Civian JT, Brennan RT, Dottolo AL, Krupat E. Experiencing the culture of academic medicine: gender matters, a national study. J Gen Intern Med 2013; 28 (02) 201-207
8 Madera JM, Hebl MR, Martin RC. Gender and letters of recommendation for academia: agentic and communal differences. J Appl Psychol 2009; 94 (06) 1591-1599
9 Schmader T, Whitehead J, Wysocki VH. A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles 2007; 57 (7-8) 509-514
10 Trix F, Psenka C. Exploring the color of glass: letters of recommendation for female and male medical faculty. Discourse Soc 2003; 14 (02) 191-220
11 Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ. Differences in words used to describe racial and gender groups in Medical Student Performance Evaluations. PLoS ONE 2017; 12 (08) e0181659
12 Isaac C, Chertoff J, Lee B, Carnes M. Do students' and authors' genders affect evaluations? A linguistic analysis of Medical Student Performance Evaluations. Acad Med 2011; 86 (01) 59-66
13 Lachman SJ, Bass AR. A direct study of halo effect. J Psychol 1985; 119 (06) 535-540
14 Gibson JL, Gore JS. Is he a hero or a weirdo? How norm violations influence the halo effect. Gend Issues 2016; 33 (04) 299-310
15 Messner AH, Shimahara E. Letters of recommendation to an otolaryngology/head and neck surgery residency program: their function and the role of gender. Laryngoscope 2008; 118 (08) 1335-1344
16 Gauer JL, Jackson JB. Association between the Medical College Admission Test scores and Alpha Omega Alpha Medical Honors Society membership. Adv Med Educ Pract 2017; 8: 627-632
17 Cuddy MM, Swanson DB, Clauser BE. A multilevel analysis of examinee gender and USMLE step 1 performance. Acad Med 2008; 83 (10, Suppl): S58-S62
18 Case SM, Becker DF, Swanson DB. Performances of men and women on NBME Part I and Part II: the more things change…. Acad Med 1993; 68 (10, Suppl): S25-S27
19 Lypson ML, Ross PT, Hamstra SJ, Haftel HM, Gruppen LD, Colletti LM. Evidence for increasing diversity in graduate medical education: the competence of underrepresented minority residents measured by an intern objective structured clinical examination. J Grad Med Educ 2010; 2 (03) 354-359
20 Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE step 1 scores to grant internal medicine residency interviews. Acad Med 2001; 76 (12) 1253-1256
21 Tekian A, Han Y, Hruska L, Krainik AJ. Do underrepresented minority medical students differ from non-minority students in problem-solving ability?. Teach Learn Med 2001; 13 (02) 86-91
22 Xierali IM, Nivet MA, Wilson MR. Current and future status of diversity in ophthalmologist workforce. JAMA Ophthalmol 2016; 134 (09) 1016-1023
23 Brothers TE, Wetherholt S. Importance of the faculty interview during the resident application process. J Surg Educ 2007; 64 (06) 378-385
24 Stohl HE, Hueppchen NA, Bienstock JL. The utility of letters of recommendation in predicting resident success: can the ACGME competencies help?. J Grad Med Educ 2011; 3 (03) 387-390
25 Daly KA, Levine SC, Adams GL. Predictors for resident success in otolaryngology. J Am Coll Surg 2006; 202 (04) 649-654
26 Rim Y. How reliable are letters of recommendation?. J Higher Educ 1976; 47 (04) 437-445