Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience

Abstract

Purpose The aim of the study is to demonstrate whether resident performance on the United States Medical Licensing Exam (USMLE) Step 1 and the Ophthalmology Knowledge Assessment Program (OKAP) correlates to American Board of Ophthalmology Written Qualifying Examination (ABO-WQE).

Design This is a retrospective cohort study.

Participants Forty-one residents who completed their postgraduate training between 1999 and 2011 from our institution were included in the study.

Methods Percentiles on the USMLE Step 1, OKAP, and ABO-WQE first attempt were recorded and analyzed for possible correlations among scores across examinations using Pearson rank correlation, chi-square analysis, Fisher exact test, and Student t-test.

Results The percentile score on the USMLE Step 1 was the only standardized test that had a statistically significant association with the percentile score on the ABO-WQE (Pearson rank coefficient = 0.45, p = 0.01). Not achieving a score of 20th percentile on the OKAP examination among second-year residents had the highest statistical association with failing the ABO-WQE on first attempt (p = 0.001) with an odds ratio of 20.4, and the relative risk of failing the ABO-WQE was 11.3; however, the relative risk of passing if the cutoff was achieved was only 1.8, and a positive predictive value for failing the ABO-WQE on first attempt of 0.47.

Conclusions Percentile scores on the OKAP are not indicative of percentile scores on the ABO-WQE. While OKAP examinations in the second and third year appear to separate out residents at risk of failing the ABO-WQE, there is a large disparity between the risk of passing and failing which is likely due to the inadequate prognostic value of the test. It is likely OKAP examinations may be a useful tool to identify residents who are at low risk for failing the ABO-WQE on first attempt, but much less useful for identifying residents who are at high risk for failing the ABO-WQE.

Keywords

USMLE - OKAP - written qualifying exam - resident - board

• References

• 1 Graff JM, Oetting TA, Lee AG. The Ophthalmic Knowledge Assessment Program (OKAP) Examination and global evaluation of resident performance. J Acad Ophthalmol 2008; 1 (1) 20-24
• 2 Chen PP, Bhandari A. Resident performance on the Ophthalmic Knowledge Assessment Program and the Written Qualifying Examination for board certification. J Acad Ophthalmol 2010; 3 (1) 11-14
• 3 Johnson GA, Bloom JN, Szczotka-Flynn L, Zauner D, Tomsak RL. A comparative study of resident performance on standardized training examinations and the American Board of Ophthalmology Written Examination. Ophthalmology 2010; 117 (12) 2435-2439
• 4 Lee AG, Oetting TA, Blomquist PH , et al. A multicenter analysis of the ophthalmic knowledge assessment program and American Board of Ophthalmology written qualifying examination performance. Ophthalmology 2012; 119 (10) 1949-1953
• 5 O'Day DM, Li C. First-time failure rates of candidates for board certification: an educational outcome measure. Arch Ophthalmol 2008; 126 (4) 548-553