Evaluation of Ophthalmology Residency Program Web Sites

Abstract

Objective This article evaluates the current availability of information on residency program Web sites that may be of interest to residency applicants.

Design Cross-sectional analysis of 117 Accreditation Council for Graduate Medical Education (ACGME)-accredited ophthalmology residency program Web sites from October 17, 2018 to December 17, 2018.

Methods Programs were identified through the ACGME-specialty search, and the URL links provided by the program were evaluated for directing the user to the institution, department, or residency program. If a link was not functional or not provided, programs were identified through a Google search. Web sites were evaluated for having information on how to apply, application requirements and eligibility, and providing a link to the San Francisco (SF) Match. Educational content included: didactic instruction, rotation overview, research requirement information, active and/or past research, institutional conferences, academic conferences, call schedules, and charitable work. Recruitment content included current residents and faculty and their biographies, graduate placement, salary, benefits, surrounding area information, message from the Program Director, Chair, and/or Chief Residents, surgical statistics, affiliated hospital information, selection criteria, and association links. Data was stratified by program size, geographic region, and top 20 hospitals in ophthalmology by the U.S. News & World Report.

Results Nonfunctional links were provided by 16.2% of programs, and 3.4% did not provide a link. Application instructions were presented by 83% of programs and 77% provided a link to the SF Match. Greater than 80% provided didactic instruction, rotation overviews, research expectations of residents, and faculty and resident listings. Up to half of programs listed information about application requirements and eligibility, call schedules, surgical statistics, academic conferences, charitable work, salary, and selection criteria. A message to applicants was displayed by 48% of programs, and 16% of programs provided association links. Programs with more than 15 residents provided more educational content than programs with less than 12 (p = 0.042). Differences in recruitment content were observed among programs in the Northeast and West, (p = 0.003) and programs in the South and West (p = 0.037). No other differences were observed.

Conclusion The Internet is frequently consulted during the residency application process. Many programs provide basic information, but this can be further optimized to maximize the utility of residency Web sites.

Keywords

ophthalmology residency - ophthalmology Web site - application - accessibility - Internet resource

Ophthalmology residency applicants likely use online resources when researching potential programs. Although there have been no surveys specifically analyzing how prospective residents in this field utilize online resources, surveys conducted within other specialties have shown that a residency's Web site influences decisions to apply or interview at a program.[1] [2] [3] There are several ways medical students can gain information about residency programs other than program Web sites, such as faculty resources, other medical students and residents, visiting student rotations, and contacting a program for information. However, residency program Web sites are a fast, easily accessible resource to access information about programs.

The availability and content of residency Web sites has been established in several other specialties and fellowships, including ophthalmic plastic and reconstructive surgery.[4] [5] [6] [7] [8] [9] [10] [11] [12] [13] To the best of our knowledge, this is the first such report for ophthalmology. The purpose of this study is to evaluate the current availability of information on residency program Web sites that may be of interest to residency applicants and identify potential areas of improvement related to education and recruitment. We also analyzed the accessibility of program information through the Accreditation Council for Graduate Medical Education (ACGME) program search and evaluated whether programs provided application instructions. We hypothesized there would be several criteria not widely addressed by all programs.

Methods

We utilized the ACGME's comprehensive list of accredited programs, accessed on October 17, 2018.[14] The number of programs listed was 119. Two programs did not have a functioning residency Web site at the time of the study and were excluded. The criteria for this study was determined by the authors and based on criteria of similar studies and surveys.[3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]

Web Site Accessibility from ACGME Site and Application Information

Each program was assessed for the presence of a URL link that directs the user to the program's Web site, defined as a functional link. Nonfunctional links were defined as directing the user to a nonworking or unrelated webpage. If both these criteria were present, then we evaluated whether the link was directed to the institution, department, or residency program. If no link was provided or the link was nonfunctional, a Google search was conducted by inputting the program's name, followed by “ophthalmology residency program,” to locate the program's Web site. We evaluated whether Web sites provided instructions on how to apply to the program, application and eligibility requirements, and a link to the San Francisco (SF) Match site.

Education

Web sites were evaluated on the presence or absence of information regarding program education. Criteria included didactic information, rotation overview, research requirements for residents, active and/or past research, call schedule, institutional conferences, academic conferences, and charitable work such as volunteering or working at free clinics.

Recruitment

Web sites were evaluated on information regarding recruitment. Criteria included program description, list of current residents and biography (medical school attended, internship, etc.), alumni information, faculty listing and biography (medical school, residency, research interests, etc.), salary, benefits, surgical statistics, and affiliated hospital information. We also evaluated whether program Web sites provided direct information or provided a link to information about the surrounding area. In addition, a message from the Program Director, Chair, or Chief Residents was highlighted in this category.

Data Stratification

Residency programs were stratified based on geographic region, program size, and Top 20 program classification, as ranked by the U.S. News and World Report. Geographic region was designated according to the United States Census Bureau regions: Midwest, Northeast, South, and West. One program not included in any census region was excluded.[15] Program size was determined by the total number of residents accredited by the ACGME. Large programs were considered to have greater than 15 total residents and small programs less than 12. In 2018, the U.S. News and World Report ranked 12 hospitals and recognized 8 as being “High Performing” hospitals for ophthalmology.[16] Collectively, these 20 programs are referred to as “top programs” for the purposes of this study. Mann–Whitney U tests were conducted for top program and program size stratifications, and a Kruskal–Wallis test was conducted for geographic region stratification. When a statistically significant difference was observed for region, a post hoc Dunn's test was used to determine which regions differed from each other. Statistical analyses were conducted utilizing Stata version 15.0 (StataCorp, College Station, TX). Significance was set at a p-value of < 0.05 for all tests.

Results

We found 119 ACGME-accredited ophthalmology residency programs through the ACGME specialty search.[14] Of these programs, two did not have a functional Web site link and could not be found in a Google search. Therefore, 117 programs were included. As seen in [Table 1], 80% of programs provided a link to the institution, department, or directly to the residency program, and 19% of programs provided either a nonfunctional link or no link at all. Less than half of programs provided application and eligibility requirements ([Table 2]).

Table 1 Percentage of programs that did not provide a link or had nonfunctional link

	Percentage of programs
No link	3.4
Nonfunctional link	16.2
Link to institution	25.6
Link to department	44.4
Link to residency program	10.3

Note: If a functional link was provided, the webpage location of where the link was directed was recorded.

Table 2 Percentage of programs providing each criterion regarding application information

	Percentage of programs
How to apply	83
Application requirements/Eligibility	43
Link to San Francisco (SF) Match	77

Education

Analyzed Web sites contained a variable amount of content pertaining to resident recruitment ([Table 3]). Didactic instruction (89%), rotation overview (85%), and expectations for resident research (86%) were three of the most presented metrics when assessing both education and recruitment. There was no difference in the educational content between regions or top program designations, but a difference was observed when stratifying by program size ([Table 5]). Larger programs provided more content than smaller programs, p = 0.042.

Table 3 Percentage of programs providing each criterion regarding total education content

Education content
Information	Percentage of programs
Didactic instruction	89
Rotation overview	85
Research requirement expectations	86
Active/Past research	67
Institution conference	54
Academic conference	43
Call schedule	50
Charitable work	17

Recruitment

Analyzed Web sites contained a variable amount of content pertaining to resident recruitment ([Table 4]). The top two categories were current resident (82%) and faculty (86%) listings, but 73 and 74% of Web sites provided biographical information about the current residents and faculty, respectively. There was a difference observed among region-stratified groups ([Table 5]). A Dunn's post hoc test revealed the mean recruitment content presented was different between programs in the Northeast and West (p = 0.003) and programs in the West and South (p = 0.037). There was no difference observed between the West and Midwest (p = 0.083), South and Midwest (p = 0.337), South and Northeast (p = 0.095), and Midwest and Northeast (p = 0.051).

Table 4 Percentage of programs providing each criterion regarding total recruitment content

Recruitment content
Information	Percentage of programs
Current residents	82
Biography of current residents	73
Career placement of resident alumni	50
Faculty listing	86
Biography of faculty	74
Salary	40
Benefits	55
Surrounding area information	39
Surgical statistics	26
Message from Program Director (PD), Chair, and/or Chiefs	48
Affiliated hospital information	58
Selection criteria	20
Association links	16

Table 5 Stratified analysis comparing mean (SD) education and recruitment content with respects to region, ranked programs, and residency size

Program characteristics	Number of programs	Number of 9 educational items addressed, mean (SD)	p-Value	Number of 13 recruitment items addressed, mean (SD)	p-Value
Region
West	17	4.9 (1.6)	p = 0.963	5.3 (2.8)	p = 0.045
South	38	5.0 (1.6)		6.9 (2.1)
Midwest	30	4.8 (1.8)		6.6 (2.2)
Northeast	31	5.0 (1.4)		7.3 (2.1)
Ranked
Top 20	20	5.3 (1.8)	p = 0.294	6.7 (2.3)	p = 0.698
Not top 20	97	4.8 (1.6)		6.6 (2.3)
Residency size
≥  15 residents	41	5.3 (1.6)	p = 0.042	7.1 (1.9)	p = 0.117
≤  12 residents	76	4.7 (1.6)		6.4 (2.5)

Abbreviation: SD, standard deviation.

Note: Mann–Whitney U test for ranked and residency size and Kruskal–Wallis for region. Significance set at p < 0.05.

Discussion

The Internet is frequently consulted when gathering information regarding residency opportunities.[1] In a survey of emergency medicine (EM) applicants, 78% of respondents reported that EM residency Web sites influenced their decision to apply to a program, and 41% reported not applying to a program because of the quality of the Web site.[3] In 2003, Mayo et al reported 83% of ophthalmology programs had departmental Web sites,[17] but the usage of the Internet has evolved since that time. Begaj et al has examined the comprehensiveness of academic ophthalmology department sites,[18] but we only analyzed factors that may be of concern to prospective trainees in application decision-making. Our goal was to conduct this study from a student-centered approach to identify areas of improvement to assist programs in developing more comprehensive Web sites.

The surveyed features were chosen based on similar studies within other specialties and based on studies specifically surveying features important to residency applicants of other specialties.[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [19] A survey of 87 plastic and reconstructive surgery applicants asked whether certain Web site features were deemed important. Of these features, career and fellowship placement (89%), current resident information (89%), faculty profiles (95%), and resident research (84%) were regarded as important.[19] In a survey of 70 radiology applicants, faculty and department research interests were identified as “necessary” information by 40% of respondents and “desirable” information by 50% of respondents for a residency program Web site.[20] A survey of 126 emergency resident applicants found that 57% of respondents reported alumni information as useful on a residency Web site.[3] These surveys have also addressed other features of this study that were found to be less important, such as surrounding area information, salary and benefits, and the presence functional URL links.[3] [19] [20] We chose to report findings on all of these features; it is ultimately up to program leaders to decide what may be important to present on their Web sites, but this study may be useful in making this determination.

Regarding education, more than 85% of programs provided information about didactics, rotations, and research requirements. However, these programs did not equally address the research interests of the institution or potential conferences that are regularly attended. Providing this information gives applicants an opportunity to match their interests with that of the institution. In a survey of radiology applicants, faculty and department research interests were identified as desirable information for a residency program Web site.[20] Call is also an important part of training and resident life, yet half of programs provided a call schedule. Similar findings are reported in otolaryngology and orthopedic residency and orthopedic trauma fellowship Web sites.[4] [6] [8]

Surgical volume was identified as a factor influencing rank order lists in a survey of ophthalmology residency applicants.[21] It is an integral part of training and can have implications on future career paths, yet 26% of programs provided surgical statistics. However, career placement of resident alumni may provide insight into career paths of residents training at an institution, and this was provided by about half of programs. A possible explanation for the lack of information provided may be due to the competitive nature of the specialty, seen by the increase in the number of applications submitted,[22] and could also explain why 20% of programs presented selection criteria.

Among data-stratified groups, we found that larger residency programs provided more information related to education, similar to the findings of the otolaryngology residency and orthopedic trauma fellowship sites.[6] [8] We also found that programs in the Northeast and South provided more information related to recruitment than programs in the West. However, our study did not equally weigh all the education and recruitment factors assessed. For example, providing surgical statistics and surrounding area information were regarded as being equal. Therefore, although statistical significance was observed, the relative importance of the factors was not considered in this determination.

There were several additional limitations to this study. First, although the utilization of residency Web sites has been established within other specialties,[1] [2] [3] the actual utility and importance of these Web sites within ophthalmology remains unknown. We acknowledge that these Web sites may not be the primary source of information for applicants, and applicants may seek out information using other methods. Since the criteria in this study were determined by the authors of this article and based on criteria established in other similar studies,[4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] future surveys could better assess how program Web sites were used in the ophthalmology residency application process. Second, this study only analyzed the presence or absence of objective criteria to minimize the likelihood of error and did not consider the ease of navigation or design aesthetic. Introducing stylistic criteria, such as webpage organization, would introduce bias and eliminate the ability to reproduce the findings. Lastly, this study was conducted within a 2-month period, so the possibility of alteration of Web sites within or after this period exists.

In conclusion, there are several areas that ophthalmology residency program Web sites can improve upon to better answer applicant questions prior to applying and interviewing at programs. Although more than 85% of programs addressed didactic instruction, rotation overviews, and research requirement information, many programs fail to address important aspects of training, such as call responsibilities, surgical statistics, and research of the institution. Since applicants often turn to the Internet when researching programs, increasing the availability of this information may assist applicants throughout the application process.

Conflict of Interest

None declared.

• References

• 1 Chu LF, Young CA, Zamora AK. , et al. Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs. Anesth Analg 2011; 112 (02) 430-439
• 2 Embi PJ, Desai S, Cooney TG. Use and utility of Web-based residency program information: a survey of residency applicants. J Med Internet Res 2003; 5 (03) e22
• 3 Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs' web sites important to student applicants. Acad Emerg Med 2005; 12 (01) 89-92
• 4 Shaath MK, Yeranosian MG, Ippolito JA, Adams MR, Sirkin MS, Reilly MC. Evaluation of the content and accessibility of web sites for accredited orthopaedic trauma surgery fellowships. J Bone Joint Surg Am 2018; 100 (09) e60
• 5 Huang BY, Hicks TD, Haidar GM, Pounds LL, Davies MG. An evaluation of the availability, accessibility, and quality of online content of vascular surgery training program websites for residency and fellowship applicants. J Vasc Surg 2017; 66 (06) 1892-1901
• 6 Davidson AR, Loftis CM, Throckmorton TW, Kelly DM. Accessibility and availability of online information for orthopedic surgery residency programs. Iowa Orthop J 2016; 36: 31-36
• 7 Wakefield DV, Manole BA, Jethanandani A. , et al. Accessibility, availability, and quality of online information for US radiation oncology residencies. Pract Radiat Oncol 2016; 6 (03) 160-165
• 8 Svider PF, Gupta A, Johnson AP. , et al. Evaluation of otolaryngology residency program websites. JAMA Otolaryngol Head Neck Surg 2014; 140 (10) 956-960
• 9 Skovrlj B, Silvestre J, Ibeh C, Abbatematteo JM, Mocco J. Neurosurgery residency websites: a critical evaluation. World Neurosurg 2015; 84 (03) 727-733
• 10 Hansberry DR, Bornstein J, Agarwal N, McClure KE, Deshmukh SP, Long S. An assessment of radiology residency program websites. J Am Coll Radiol 2018; 15 (04) 663-666
• 11 Ashack KA, Burton KA, Soh JM. , et al. Evaluating dermatology residency program websites. Dermatol Online J 2016; 22 (03) 13030/qt7rx3j2dn
• 12 Homer N, Yoon MK. Evaluation of the American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) fellowship program website content and quality. Ophthal Plast Reconstr Surg 2017; 33 (06) 471-473
• 13 Stoeger SM, Freeman H, Bitter B, Helmer SD, Reyes J, Vincent KB. Evaluation of general surgery residency program websites. Am J Surg 2019; 217 (04) 794-799
• 14 Accreditation Council for Graduate Medical Education (ACGME) Advanced Program Search. [Website]. 2018 . Available at: https://apps.acgme.org/ads/Public/Programs/Search . Accessed October 17, 2018
• 15 Regions. 2010 . Available at: https://www.census.gov/geo/reference/webatlas/regions.html . Accessed March 3, 2019
• 16 US News and World Report Best Hospitals for Ophthalmology [Report]. Available at: https://health.usnews.com/best-hospitals/rankings/ophthalmology . Accessed December 18, 2018
• 17 Mayo GL, Lindhorst GC, Rosende C. American ophthalmology graduate medical education and the web: current state of internet resource utilization. Am J Ophthalmol 2003; 135 (05) 708-709
• 18 Begaj T, Helmy O, Leeman S, Schaal S. The current online face of U.S. academic ophthalmology. J Acad Ophthalmol 2018; 10 (01) e114-e121
• 19 Chen VW, Hoang D, Garner W. Do websites provide what applicants need? Plastic Surgery residency program websites versus applicant self-reported needs. Plast Reconstr Surg Glob Open 2018; 6 (10) e1900
• 20 Deloney LA, Perrot LJ, Lensing SY, Jambhekar K. Radiology resident recruitment: a study of the impact of web-based information and interview day activities. Acad Radiol 2014; 21 (07) 931-937
• 21 Yousuf SJ, Kwagyan J, Jones LS. Applicants' choice of an ophthalmology residency program. Ophthalmology 2013; 120 (02) 423-427
• 22 Ophthalmology Residency Match Summary Report; 2018 . Available at: https://www.sfmatch.org/PDFFilesDisplay/Ophthalmology_Residency_Stats_2018.pdf . Accessed December 25, 2018