Subscribe to RSS
DOI: 10.1055/s-0040-1718569
Ophthalmology Program Directors' Perspectives on the Impact of the United States Medical Licensing Examination Step 1 Change to Pass-Fail Scoring
Abstract
Purpose This article investigates the perspectives of ophthalmology residency program directors (PDs) regarding the impact of the United States Medical Licensing Examination (USMLE) Step 1 change from graded to pass-fail scoring on ophthalmology resident selection and medical education.
Methods The PDs of all United States ophthalmology residency programs accredited by the Accreditation Council for Graduate Medical Education were identified using a public, online database. An anonymous web-based survey constructed using REDCap was emailed to each PD in February 2020.
Results Surveys were completed by 64 (54.2%) PDs, with the majority (81.2%) disagreeing with the change to pass-fail scoring. The majority of PDs believe this change will negatively impact the ability to evaluate residency applicants (92.1%) and achieve a fair and meritocratic match process (76.6%), and will decrease medical students' basic science knowledge (75.0%). The factors identified most frequently by PDs as becoming more important in evaluating residency applicants as a result of the Step 1 scoring change include clerkship grades (90.6%), USMLE Step 2 Clinical Knowledge score (84.4%), and a rotation in the PD's department (79.7%). The majority of PDs believe the Step 1 grading change to pass-fail will benefit applicants from elite medical schools (60.9%), and disadvantage applicants from nonelite allopathic schools (82.8%), international medical graduate applicants (76.6%), and osteopathic applicants (54.7%).
Conclusion The majority of ophthalmology PDs disagree with the change in USMLE Step 1 scoring from graded to pass-fail and believe this change will negatively impact the ability to evaluate residency applicants and achieve a fair and meritocratic match process, and will decrease medical students' basic science knowledge.
#
Keywords
ophthalmology match - Step 1 - United States Medical Licensing Examination - USMLE - pass-fail - Step 2 - Step 2 Clinical Knowledge - residency match - ophthalmology resident - medical student - residency program director - medical educationThe Ophthalmology Residency Matching Program (OMP), founded by the Association of University Professors of Ophthalmology in 1977, serves as the application process for ophthalmology residency.[1] The process separates the ophthalmology match from the National Resident Matching Program (NRMP), which manages the matching process for almost all other medical specialties. The application deadline and match day of the OMP are earlier than those of the NRMP.[1] [2]
An important component of the ophthalmology residency application involves the United States Medical Licensing Examinations (USMLE), a series of assessments sponsored by the National Board of Medical Examiners (NBME) that U.S. allopathic (MD) medical students must pass to obtain a medical license. The first of these series of examinations, the USMLE Step 1, measures basic science knowledge and has traditionally been a graded examination taken after the second year of medical school and before the start of clinical rotations. The remaining USMLE examinations taken in medical school, the graded Step 2 Clinical Knowledge (CK) and the pass-fail Step 2 Clinical Skills, are usually taken after the third year of medical school. While Step 1 has historically been one of the most important metrics for residency selection,[3] [4] [5] [6] [7] nearly half (46%) of ophthalmology residency applicants did not report their Step 2 scores in 2011, potentially due to ophthalmology's early match.[8]
Ophthalmology residency programs receive hundreds of applicants per residency position offered, with recent years demonstrating an upward trend in the number of programs to which each applicant applied.[9] The graded USMLE Step 1 examination has provided programs with a standardized measurement to compare a large number of applicants from different backgrounds in medical education. The competitive nature of matching into ophthalmology is evidenced by the higher mean Step 1 score of applicants who matched into ophthalmology compared with that of all applicants who matched into any residency (245 vs. 233 most recently in 2018, with a similar score difference between the two groups since the OMP first made these data available in 2011).[10] [11] [12] Common criticisms of Step 1's role in evaluating applicants include the overemphasis of a 1-day examination, Step 1 being originally designed as a pass-fail examination for licensure and not for the purpose of comparing applicants, a potential negative impact on student wellness, residency programs forgoing holistic application review which may disadvantage underprivileged applicants, Step 1 being a generally unreliable metric for resident performance, and medical schools targeting their basic science curricula to the Step 1 examination rather than clinical application.[6] [13] [14] [15] [16]
On February 12, 2020, after a year of updates and deliberation, the NBME announced the change of Step 1 to a pass-fail scoring system to be implemented no earlier than 2022.[14] [17] The purpose of our study is to investigate the perspectives of ophthalmology residency program directors (PDs) regarding the impact of the USMLE Step 1 examination change from graded to pass-fail scoring on resident selection and medical education.
Methods
The study was approved by the Institutional Review Board of the Penn State College of Medicine and conducted in accordance with the Declaration of Helsinki. The survey ([Table 1]) was constructed using the secure online application REDCap hosted at the Penn State College of Medicine.[18] The survey was designed based on a review of existing literature,[3] [4] [5] [6] [7] [8] [9] [14] and questions arising from discussion with colleagues involved in medical education. The survey was pretested by members of the study team and we collected feedback on the content, clarity, formatting, and style of the survey. The survey was refined to arrive at its current format. Contact information for each PD was obtained using the public online database FREIDA™ of the American Medical Association.[19] An email with the study description, an invitation to participate in the study, and a secure link was sent to the PD of each of the 118 ophthalmology residency programs accredited by the Accreditation Council for Graduate Medical Education on February 28, 2020. Responses were collected automatically and managed under a deidentified record number generated by REDCap. Using the REDCap system, nonresponders were identified anonymously and were sent reminder emails.
#
Results
Surveys were completed by 64 of 118 (54.2%) PDs. The median age of respondents was 50 (with a standard deviation of 9.5 years), 73.4% of respondents were male, 26.6% were female, and 70.3% self-identified as white, 23.4% as Asian, and 6.3% as Hispanic, Latino, or of Spanish origin.
A majority (52, 81.2%) of PDs disagree with the change in Step 1 from graded to pass-fail scoring, with 7 (10.9%) neutral on the decision and 5 (7.8%) agreeing with the change. The same majority (52, 81.2%) disagree that the decision for the change had been made in a transparent fashion with adequate involvement of all stakeholders, with 7 (10.9%) neutral and 5 (7.8%) agreeing. Regarding the utility of the current Step 1 (that is, Step 1 with a graded score), 50 (78.1%) agree that the test adequately measures basic science knowledge with 13 neutral (20.3%) and 1 (1.6%) disagreeing. PDs were more divided on the utility of the graded Step 1 to adequately measure medical knowledge (37.5% agree, 29.7% neutral, 32.8% disagree), the work ethic of an applicant (42.2% agree, 25.0% neutral, 32.8% disagree), and the ability of an applicant to succeed in medicine (34.4% agree, 37.5% neutral, 31.3% disagree). Given the change in Step 1 from graded to pass-fail scoring, 39 (60.9%) PDs support the movement of medical schools to a graded (rather than a pass-fail) preclinical curriculum as a means of now evaluating a student's basic science knowledge, with 14 (21.9%) unsure and 11 (17.2%) disagreeing.
[Table 2] summarizes PDs' perspectives on how the Step 1 scoring change will impact the ophthalmology resident selection process, medical student education, and applicant groups based on their medical education background. A majority of PDs believe that the Step 1 scoring change will make it more difficult to select which applicants to interview and rank (59, 92.1%), and to achieve a fair and meritocratic resident selection process (49, 76.6%). A majority of PDs also believe that the Step 1 scoring change will have a negative impact on medical students' basic science knowledge (42, 75.0%), no impact on clinical knowledge (35, 54.7%) and general well-being (38, 59.3%), and a negative impact on the ability to create better future physicians (35, 54.7%). Most PDs believe the change will provide an advantage to medical students who attend an elite medical school (39, 60.9%), but will disadvantage medical students who attend a nonelite allopathic medical school (53, 82.8%), osteopathic medical students (35, 54.7%), and international medical graduates who attend medical school outside the United States (49, 76.6%).
PDs also report that, as a result of the change in Step 1 scoring, they expect applicants will apply to a greater number of residency programs (37, 57.8%), attend more rotations in ophthalmology including away rotations (38, 59.4%), and engage in more research (27, 42.2%).
[Table 3] summarizes PDs' perspectives on how the Step 1 scoring change will impact the relative importance of residency application factors. In summary, PDs believe that all application components other than the Step 1 score will be considered more highly after institution of the Step 1 scoring change. The factors identified most frequently by PDs as becoming more important in evaluating residency applicants as a result of the Step 1 scoring change include clerkship grades (58, 90.6%), USMLE Step 2 CK score (54, 84.4%), and a rotation in the PD's department (51, 79.7%).
#
Discussion
A large majority of the ophthalmology PDs who responded to our survey disagree with the change from graded to pass-fail scoring of the USMLE Step 1 examination (81.2%) and do not believe the decision for the change was made in a transparent fashion with adequate involvement of all stakeholders (81.2%). Most PDs also have concerns that the change will impair the ability to evaluate which candidates to interview and rank (92.1%), and to achieve a fair and meritocratic match process (76.6%). The results of our study support existing literature that suggests Step 1 has historically played a significant role in ophthalmology resident selection,[3] [4] [5] [6] [7] and indicate that its removal is perceived by PDs as increasing the challenges associated with evaluating and comparing applicants. Furthermore, most PDs believe that a graded Step 1 adequately measures basic science knowledge (78.1%) and that the change to pass-fail scoring will adversely affect medical students' basic science knowledge (75.0%).
The results of the current study highlight the important role that standardized examinations have historically played in the evaluation of applicants, particularly for highly competitive education programs. Much like the Medical College Admissions Test used for medical school admission, Step 1 has been perceived by PDs to be helpful in comparing applicants. While these standardized tests have important limitations, particularly with regards to underrepresented minorities and those of lower socioeconomic status scoring lower,[13] [14] [20] they have served as useful tools to be considered carefully in conjunction with other factors. With the change in Step 1 scoring to pass-fail, programs may simply adopt the Step 2 CK graded score as a replacement, a potential utilization acknowledged by the USMLE[21] and supported by the results of the current study, in which a large majority of PDs (84.4%) indicate its increased importance; in fact, the Step 2 CK graded score was the application factor with the most PDs (48.4%) ranking it as “significantly more important.”
Currently, medical students who score poorly on the USMLE Step 1 are encouraged to demonstrate significant improvement on the Step 2 CK to have a competitive application for residency. With Step 2 CK as the only graded standardized medical student examination on the ophthalmology residency application form, there is likely to be increased pressure on medical students to perform well on the Step 2 CK, which may negate the “well-being” benefit desired from the Step 1 change to pass-fail scoring. Furthermore, given that the Step 2 CK is taken after the third year of medical school and ophthalmology's early match system, those interested in ophthalmology may only discover how competitive of a score they received on this examination shortly before applying to residency programs. Available data suggest that more than half of current ophthalmology residency applicants do not submit their Step 2 CK score as part of their application.[8] If programs begin to evaluate Step 2 CK, it may be beneficial for the OMP to push back its early match deadline.
One major concern illustrated by the results of the current study is that removal of Step 1 as a standardized metric for evaluating applicants may create an unfair bias benefitting applicants from privileged medical education backgrounds. The majority of PDs believe the Step 1 scoring change will provide an advantage to applicants from elite medical schools (60.9%), and will disadvantage applicants from nonelite allopathic schools (82.8%), international medical graduate applicants (76.6%), and osteopathic applicants (54.7%), applicant groups which already have a more difficult time matching into ophthalmology compared with those who attend an elite medical school.[4] [5] [7] [12] A large majority of surveyed PDs believe that the Step 1 scoring change will make it more difficult to achieve a fair and meritocratic resident selection process (45, 76.6%). Highly regarded medical schools, in addition to their name recognition, may be more likely to offer better and more research opportunities for medical students and letters of recommendation from well-known faculty, and all of these factors may now play larger roles in residency applicant selection. While applicants from elite medical schools may be perceived as having had a more rigorous medical education, standardized examinations such as a graded USMLE Step 1 can help “equalize” candidates from diverse medical institutions and allow applicants from nonelite medical schools to demonstrate their academic abilities. Thus, the change in Step 1 scoring may even impact premedical students, by placing increased emphasis on attending the best medical school possible at the expense of foregoing financial aid from public or less-prestigious schools.
Further highlighting that the Step 1 scoring change may benefit applicants with more resources, the majority of PDs reported they will place increased importance on applicants attending a rotation in the PD's department (79.7%) and believe that applicants will attend more ophthalmology rotations including away rotations (59.4%) and apply to more programs (57.8%). Attending away rotations, applying to more programs, and partaking in additional interviews is not just stressful and time-intensive, but financially burdensome, especially considering that ophthalmology applicants in the 2018 to 2019 cycle spent an average of $5,704 for applications and interviews.[22] A potential initiative for programs is to implement video conference interviewing,[23] which can help alleviate the financial burden on applicants, especially those attending nonelite medical schools who may feel compelled to apply to more programs to increase their success in matching in ophthalmology. Another initiative that warrants discussion is a limitation on the number of programs to which an applicant may apply. Given that ophthalmology is a competitive medical specialty, the mean number of programs to which an ophthalmology residency applicant applies has increased from 45 to 68 between 2006 and 2016.[9] Placing a limit on the number of programs to which an applicant may apply would allow programs to spend more time holistically reviewing each application. Further, programs would know that an applicant has genuine interest in their program, and students would experience less financial burden.
While this study highlights potential disadvantages from the change in Step 1 scoring, there are also data from our survey that support the USMLE's decision to implement the scoring change. One rationale for the scoring change was to improve the well-being of medical students; while the majority of PDs (59.3%) believe that the scoring change will result in no change in student well-being, it is worth noting that more PDs agree than disagree that this will improve student well-being (31.3% vs. 9.4%). In addition, sponsors of the USMLE argue that the examination was originally designed as a pass-fail licensing examination to ensure adequate basic science knowledge, and has little utility for comparing applicants or clinical application.[14] While the majority of surveyed PDs agree that Step 1 adequately measures basic science knowledge, there is divide on whether the examination is useful in assessing applicants' clinical knowledge, ability to succeed in medicine, and work ethic. Thus, the perspectives of the PD respondents in our survey support existing literature that the Step 1 examination itself may not necessarily have the utility to predict success of future residents but is limited to use as a standardized examination for assessing science knowedge.[11] [12] [13] [14]
Of note, only 4 (6.3%) of the ophthalmology PD respondents self-identified as Hispanic, Latino, or of Spanish origin, with none self-identifying as black or African American. Although a 2020 study reported various demographic data for all PDs in the United States, race and ethnicity of the PDs were not included.[24]
When interpreting our results, it is important to keep in mind that only the perspectives of ophthalmology residency PDs are assessed in the current study and the perspectives of other stakeholders may well differ. For example, the USMLE Step 1 scoring change will allow medical schools greater autonomy in structuring their curricula and teaching to clinical application, rather than board preparation. Nevertheless, the significance of our project is its focus on how the Step 1 scoring change may impact ophthalmology resident selection and, therefore, the results have important implications for medical students interested in ophthalmology and all parties involved in the ophthalmology resident selection process.
Another limitation of our study is potential responder bias. Although the majority of ophthalmology PDs elected to participate in this survey, the responding PDs may be those with particularly strong opinions regarding the topic and their perspectives may not be generalizable to all ophthalmology PDs.
In summary, a large majority of surveyed ophthalmology residency PDs disagree with the USMLE Step 1 change from graded to pass-fail scoring, and believe that, as a result, application factors other than the USMLE Step 1, such as USMLE Step 2 CK scores, clinical grades, ophthalmology rotations, letters of recommendation, personal knowledge of an applicant, and the prestige of an applicant's medical institution, will have increased significance in the resident selection process. Most PDs believe the change will negatively impact the ability to evaluate residency applicants and achieve a fair and meritocratic match process, and will decrease medical students' basic science knowledge. The majority of PDs believe the change in Step 1 from graded to pass-fail scoring will benefit applicants from elite medical schools, and disadvantage applicants from nonelite allopathic schools, international medical graduate applicants, and osteopathic applicants.
#
#
Conflict of Interest
None declared.
-
References
- 1 San Francisco Match, Overview: Ophthalmology residency match. Available at: https://sfmatch.org/SpecialtyInsideAll.aspx?id=6&typ=2&name=Ophthalmology. Accessed April 15, 2020
- 2 The National Residency Matching Program, 2–2- Main residency match calendar. Available at: https://nrmp.org/match-calendars/. Accessed April 15, 2020
- 3 The National Resident and Matching Program, Results of the 2018 NRMP program director survey. Available at: https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf. Accessed April 15, 2020
- 4 Loh AR, Joseph D, Keenan JD, Lietman TM, Naseri A. Predictors of matching in an ophthalmology residency program. Ophthalmology 2013; 120 (04) 865-870
- 5 Siatkowski RM, Mian SI, Culican SM. Association of University Professors of Ophthalmology. et al. Probability of success in the ophthalmology residency match: three-year outcome analysis of San Francisco matching program data. J Acad Ophthalmol 2018; 10 (01) e150-e157
- 6 Nallasamy S, Uhler T, Nallasamy N, Tapino PJ, Volpe NJ. Ophthalmology resident selection: current trends in selection criteria and improving the process. Ophthalmology 2010; 117 (05) 1041-1047
- 7 Driver TH, Loh AR, Joseph D, Keenan JD, Naseri A. Predictors of matching in ophthalmology residency for international medical graduates. Ophthalmology 2014; 121 (04) 974-975
- 8 Yousuf SJ, Jones LS. Ophthalmology residency match outcomes for 2011. Ophthalmology 2012; 119 (03) 642-646
- 9 Grubbs Jr JR, Mian SI. Advising students interested in ophthalmology: a summary of the evidence. Ophthalmology 2016; 123 (07) 1406-1410
- 10 The National Resident Matching Program, Charting outcomes in the match: U.S. allopathic seniors, Characteristics of U.S. allopathic seniors who matched to their preferred specialty in the 2018 main residency match. Available at: nrmp.org/wp-content/uploads/2018/06/Charting-Outcomes-In-The-Match-2018-Seniors.pdf. Accessed April 15, 2020
- 11 The National Resident Matching Program, Charting outcomes in the 2011 match: U.S. allopathic seniors, Characteristics of U.S. allopathic seniors who matched to their preferred specialty in the 2011 main residency match. Available at: nrmp.org/report-archives/. Accessed April 15, 2020
- 12 The San Francisco Match and the Association of University Professors of Ophthalmology, Ophthalmology Residency Match Report. Available at: sfmatch.org/SpecialtyINsideAll.aspx?id=6&typ=2&name=Ophthalmology#. Accessed April 15, 2020
- 13 Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med 2016; 91 (01) 12-15
- 14 Invitation Conference on USMLE Scoring (InCUS). Summary report and preliminary recommendation from the invitation conference on USMLE scoring, March 11–12. 2019 . Available at: https://usmle.org/pdfs/incbus/incus_summary_report.pdf. Accessed April 15, 2020
- 15 Black KP, Abzug JM, Chinchilli VM. Orthopaedic in-training examination scores: a correlation with USMLE results. J Bone Joint Surg Am 2006; 88 (03) 671-676
- 16 Gunderman RB, Jackson VP. Are NBME examination scores useful in selecting radiology residency candidates?. Acad Radiol 2000; 7 (08) 603-606
- 17 United States Medical Licensing Examination, Change to pass/fail score reporting for Step 1. Available at: https://www.usmle.org/incus/. Accessed April 15, 2020
- 18 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
- 19 The American Medical Association, FREIDA the AMA residency & fellowship database. Available at: https://freida.ama-assn.org/Freida. Accessed April 15, 2020
- 20 Davis D, Dorsey JK, Franks RD, Sackett PR, Searcy CA, Zhao X. Do racial and ethnic group differences in performance on the MCAT exam reflect test bias?. Acad Med 2013; 88 (05) 593-602
- 21 United States Medical Licensing Examination, Three decision to result in future changes to USMLE. Available at: https://www.usmle.org/pdfs/viewer/viewer.html?file=/pdfs/podcast/Three-decision-to-result-in-future-changes-toUSMLE.pdf?123. Accessed April 15, 2020
- 22 Venincasa MJ, Cai LZ, Gedde SJ, Uhler T, Sridhar J. Current applicant perceptions of the ophthalmology residency match. JAMA Ophthalmol 2020; DOI: 10.1001/jamaophthalmol.2020.0252.
- 23 Greenberg PB, Scott IU. et al. Video conference interviewing for the ophthalmology residency match. JAMA Ophthalmol (in press)
- 24 Kloosterboer A, Yannuzzi NA, Gedde SJ, Sridhar J. Residency program directors of United States ophthalmology programs: a descriptive analysis. Am J Ophthalmol 2020; 209: 71-76
Address for correspondence
Publication History
Received: 30 May 2020
Accepted: 03 September 2020
Article published online:
26 November 2020
© 2020. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA
-
References
- 1 San Francisco Match, Overview: Ophthalmology residency match. Available at: https://sfmatch.org/SpecialtyInsideAll.aspx?id=6&typ=2&name=Ophthalmology. Accessed April 15, 2020
- 2 The National Residency Matching Program, 2–2- Main residency match calendar. Available at: https://nrmp.org/match-calendars/. Accessed April 15, 2020
- 3 The National Resident and Matching Program, Results of the 2018 NRMP program director survey. Available at: https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf. Accessed April 15, 2020
- 4 Loh AR, Joseph D, Keenan JD, Lietman TM, Naseri A. Predictors of matching in an ophthalmology residency program. Ophthalmology 2013; 120 (04) 865-870
- 5 Siatkowski RM, Mian SI, Culican SM. Association of University Professors of Ophthalmology. et al. Probability of success in the ophthalmology residency match: three-year outcome analysis of San Francisco matching program data. J Acad Ophthalmol 2018; 10 (01) e150-e157
- 6 Nallasamy S, Uhler T, Nallasamy N, Tapino PJ, Volpe NJ. Ophthalmology resident selection: current trends in selection criteria and improving the process. Ophthalmology 2010; 117 (05) 1041-1047
- 7 Driver TH, Loh AR, Joseph D, Keenan JD, Naseri A. Predictors of matching in ophthalmology residency for international medical graduates. Ophthalmology 2014; 121 (04) 974-975
- 8 Yousuf SJ, Jones LS. Ophthalmology residency match outcomes for 2011. Ophthalmology 2012; 119 (03) 642-646
- 9 Grubbs Jr JR, Mian SI. Advising students interested in ophthalmology: a summary of the evidence. Ophthalmology 2016; 123 (07) 1406-1410
- 10 The National Resident Matching Program, Charting outcomes in the match: U.S. allopathic seniors, Characteristics of U.S. allopathic seniors who matched to their preferred specialty in the 2018 main residency match. Available at: nrmp.org/wp-content/uploads/2018/06/Charting-Outcomes-In-The-Match-2018-Seniors.pdf. Accessed April 15, 2020
- 11 The National Resident Matching Program, Charting outcomes in the 2011 match: U.S. allopathic seniors, Characteristics of U.S. allopathic seniors who matched to their preferred specialty in the 2011 main residency match. Available at: nrmp.org/report-archives/. Accessed April 15, 2020
- 12 The San Francisco Match and the Association of University Professors of Ophthalmology, Ophthalmology Residency Match Report. Available at: sfmatch.org/SpecialtyINsideAll.aspx?id=6&typ=2&name=Ophthalmology#. Accessed April 15, 2020
- 13 Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med 2016; 91 (01) 12-15
- 14 Invitation Conference on USMLE Scoring (InCUS). Summary report and preliminary recommendation from the invitation conference on USMLE scoring, March 11–12. 2019 . Available at: https://usmle.org/pdfs/incbus/incus_summary_report.pdf. Accessed April 15, 2020
- 15 Black KP, Abzug JM, Chinchilli VM. Orthopaedic in-training examination scores: a correlation with USMLE results. J Bone Joint Surg Am 2006; 88 (03) 671-676
- 16 Gunderman RB, Jackson VP. Are NBME examination scores useful in selecting radiology residency candidates?. Acad Radiol 2000; 7 (08) 603-606
- 17 United States Medical Licensing Examination, Change to pass/fail score reporting for Step 1. Available at: https://www.usmle.org/incus/. Accessed April 15, 2020
- 18 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
- 19 The American Medical Association, FREIDA the AMA residency & fellowship database. Available at: https://freida.ama-assn.org/Freida. Accessed April 15, 2020
- 20 Davis D, Dorsey JK, Franks RD, Sackett PR, Searcy CA, Zhao X. Do racial and ethnic group differences in performance on the MCAT exam reflect test bias?. Acad Med 2013; 88 (05) 593-602
- 21 United States Medical Licensing Examination, Three decision to result in future changes to USMLE. Available at: https://www.usmle.org/pdfs/viewer/viewer.html?file=/pdfs/podcast/Three-decision-to-result-in-future-changes-toUSMLE.pdf?123. Accessed April 15, 2020
- 22 Venincasa MJ, Cai LZ, Gedde SJ, Uhler T, Sridhar J. Current applicant perceptions of the ophthalmology residency match. JAMA Ophthalmol 2020; DOI: 10.1001/jamaophthalmol.2020.0252.
- 23 Greenberg PB, Scott IU. et al. Video conference interviewing for the ophthalmology residency match. JAMA Ophthalmol (in press)
- 24 Kloosterboer A, Yannuzzi NA, Gedde SJ, Sridhar J. Residency program directors of United States ophthalmology programs: a descriptive analysis. Am J Ophthalmol 2020; 209: 71-76