Evaluation of the Content of Ophthalmology Fellowship Program Web sites

Abstract

Purpose To assess the availability and content of fellowship program Web sites (FPWs) among ophthalmology subspecialties.

Design This is a cross-sectional study.

Subjects Web sites of all Association of University Professors of Ophthalmology-accredited fellowship programs in five subspecialties (i.e., surgical retina and vitreous; cornea, external disease, and refractive surgery; glaucoma; neuro-ophthalmology; and pediatric ophthalmology).

Methods FPWs were assessed for the presence of 26 key content criteria encompassing program demographics (n = 13), features (n = 10), and social life (n = 3). The presence of each content criterion as well as the content criteria groups were compared across subspecialties.

Main Outcome Measures The main outcome measured is the average percentage of key content criteria present among ophthalmology fellowship Web sites.

Results Among 266 accredited fellowship programs, 240 (90.2%) had Web sites. On average, Web sites reported 14.9 of 26 key content criteria (57.2%), 8.29 of 13 demographic criteria (63.8%), 5.84 of the 10 program features criteria (58.4%), and 0.705 of the 3 social life criteria (23.5%). Significant differences were identified among subspecialties in the presence of program description (p = 0.046), hospital affiliation (p < 0.001), names of current fellows (p = 0.004), case diversity (p = 0.001), and surgical statistics (p = 0.015). The average number of key criteria differed between subspecialties (p < 0.001).

Conclusion There is significant heterogeneity in program fellowship Web site content among ophthalmology subspecialties. Information regarding social life, such as wellness programs and community information, was largely absent across all disciplines. Addressing missing information on ophthalmology FPWs may help optimize program-applicant fit.

Keywords

fellowship - Web site - Web site content - ophthalmology fellowships

As a result of the COVID-19 pandemic, residency and fellowship programs were adapted by using virtual interviews and information sessions. A virtual format places increased emphasis on the availability and quality of online resources for applicants to prepare before an interview.[1] As a vehicle for that information, program Web sites play an important role in guiding residency and fellowship applicants' decision-making. Several studies across medical specialties have identified online presence both in Web site and social media as a determining factor in fellowship selection among new physicians.[2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]

Fellowship program Web sites (FPWs) have been assessed across medical specialties for content relevant to applicants.[14] [15] As part of this assessment, authors established criteria to describe FPW content that may be utilized as a comparative scoring method. For example, Wei et al utilized 20 distinct criteria to evaluate 147 dermatopathology, micrographic surgery and dermatologic oncology, and pediatric dermatology FPWs.[13] Ruddell et al assessed thoracic surgery program Web sites based on 25 distinct criteria.[14] Previous studies of ophthalmology trainees have identified factors that are particularly influential when applying to programs and deciding whether to subspecialize.[2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [16] [17] Many of the factors identified are ones included in previous Web site assessments, such as surgical volume and research opportunities.

Ophthalmologic FPW content has been assessed only for ophthalmic plastic and reconstructive surgery.[15] Reviewing program Web sites of the remaining ophthalmology subspecialty fellowships may provide insight into any discrepancies in the information available among subspecialties. Notably, a difference in program Web site content availability and quality between subspecialties may have an unintended effect on applicants' decisions when applying to fellowship programs. This study assesses the availability and breadth of 26 predefined content items on the Web sites of Association of University Professors of Ophthalmology (AUPO)-compliant surgical retina and vitreous; cornea, external disease, and refractive surgery; glaucoma; neuro-ophthalmology; and pediatric ophthalmology fellowship programs.

Methods

Study Design

Web sites of programs from the following subspecialty fellowships were analyzed for completeness of content: surgical retina and vitreous; cornea, external disease, and refractive surgery; glaucoma; neuro-ophthalmology; and pediatric ophthalmology. A list of AUPO-compliant ophthalmology fellowship programs within these five subspecialties was extracted from the AUPO “Programs in Compliance” Web site.[18] These subspecialties were chosen for analysis since they offer the greatest number of accredited fellowship positions annually. Ophthalmic reconstructive and plastic surgery fellowship programs were not assessed as they have been examined previously and are not supported by the AUPO Fellowship Compliance Committee.[15] Programs were assessed for the presence of a dedicated Web site or Web site section. Afterward, each Web site was scored on the presence or absence of 26 key content criteria derived from previous studies assessing Web site content of ophthalmic plastic and reconstructive surgery, dermatology, and cardiothoracic surgery fellowship Web sites.[15] [16] [17] Based on Weibull distribution modeling indicating dwell times of 10 to 20 seconds, content was considered “accessible” if it could be found within two clicks from the FPW web page.[19] Videos or documents found within two clicks from the web page were also reviewed for content criteria.

Among the 26 content criteria chosen, 13 were categorized as “program demographics,” including: description of the fellowship program, phone number of the fellowship or ophthalmology department, e-mail of the fellowship or ophthalmology department, mailing address of the fellowship or ophthalmology department, name of the fellowship director, affiliated hospital, number of positions available, names of current fellows, names of past fellows, current employers of past fellows, link to or location of application form, requirements for application, and application deadline. Ten criteria were categorized as “program features,” including: salary, employment benefits, didactic information (mention of grand rounds, lectures, bedside learning), journal club, rotation schedule (either day-to-day or semester schedule), clinic or call responsibilities, research opportunities and requirements, availability of time off/funding for conferences, case diversity, and surgical statistics. Three criteria were categorized as “social life,” including: community information/activities to do in the area, health and wellness programs, and link to a fellowship program or ophthalmology department/hospital social media page.

Statistical Analysis

Fellowship Web site presence was compared between ophthalmology subspecialties using a chi-square test. The average percentage of FPW criteria (total and by criterion cluster) present was compared across subspecialties and analyzed using the chi-square test. Chi-square tests were also used to test for nonequivalence between observed and expected values of the 26 Web site criteria between different subspecialties. Fisher's exact test was used when the assumptions for a chi-square test were not met. Analyses were performed in Microsoft Excel (Redmond, WA).

Results

Two hundred forty of 266 (90.2%) distinct fellowship programs had Web sites. There was no statistically significant difference among subspecialties in Web site presence (p = 0.255) ([Table 1]). Across all the disciplines, FPWs contained an average of 57.2% of the total criteria, 63.8% of program demographic criteria, 58.4% of program feature criteria, and 23.5% of social life criteria ([Table 2]). Between disciplines, there were significant differences in the presence of total criteria (p < 0.001), program features criteria (p < 0.001), and social life criteria (p = 0.031). Neuro-ophthalmology FPWs contained the lowest percentage of total criteria (50.8%) and program features criteria (49.2%), while glaucoma programs contained the lowest percentage of social life factors (17.2%).

Table 1 Fellowship Web site presence

	Fellowship programs with Web site, N (%)
Surgical retina and vitreous (n = 77)	70 (90.9)
Cornea, external disease, and refractive surgery (n = 50)	42 (84.0)
Glaucoma (n = 66)	58 (87.8)
Neuro-ophthalmology (n = 29)	28 (96.6)
Pediatric ophthalmology (n = 44)	42 (95.5)
	p = 0.255[ a ]

^a The p-value was calculated using chi-square test.

Table 2 Average percentage of Web site criteria by fellowship subspecialty

Web site criteria group	Total (%)	Surgical retina and vitreous (%)	Cornea, external disease, and refractive surgery (%)	Glaucoma (%)	Neuro-ophthalmology (%)	Pediatric ophthalmology (%)	p-Value[a]
All factors (n = 26)	57.2	61.0	58.3	53.5	50.8	59.3	< 0.001
Program demographics (n = 13)	63.8	66.2	63.7	60.1	62.6	65.8	0.099
Program features (n = 10)	58.4	64.0	58.1	55.5	49.2	63.1	< 0.001
Social life (n = 3)	23.5	27.1	31.0	17.2	22.6	19.0	0.031

^a The p-values were calculated using chi-square tests. Bold text indicates a p-value significant at 5% level of significance.

The presence of each individual content factor by fellowship subspecialty Web site is summarized in [Table 3]. In the program demographics cluster, significant differences between specialties were observed in program description (with neuro-ophthalmology programs most often lacking this criterion [92.9%, p = 0.046], average = 98.3%), presence of a mailing address (with glaucoma programs most often lacking this criterion [48.3%, p = 0.014], average = 61.6%), hospital affiliation (with neuro-ophthalmology programs most often lacking this criterion [85.7%, p < 0.001], average = 97.9%), name of current fellows (with pediatric ophthalmology programs most often lacking this criterion [28.6%, p = 0.004], average = 46.3%), and application requirements (with both surgical retina and vitreous, and cornea programs most often lacking this criterion [42.9%, p = 0.022], average = 50.0%). In the program features cluster, significant differences were noted for the presence of a journal club (with neuro-ophthalmology programs most often lacking this criterion [17.9%, p = 0.016], average = 42.9%), case diversity (with neuro-ophthalmology programs most often lacking this criterion [50.0%, p = 0.001], average: 79.6%), and surgical statistics (with neuro-ophthalmology programs most often lacking this criterion [10.7%, p = 0.015], average: 30.8%). No statistically significant differences were observed in the social life category ([Table 3]).

Table 3 Fellowship Web site criteria by fellowship subspecialty

Factor	Surgical retina and vitreous (n = 70), n (%)	Cornea, external disease, and refractive surgery (n = 42), n (%)	Glaucoma (n = 58), n (%)	Neuro-ophthalmology (n = 28), n (%)	Pediatric ophthalmology (n = 42), n (%)	p-Value[a]
Program demographics
Program description	70 (100)	41 (97.6)	58 (100)	26 (92.9)	41 (97.6)	0.046
Telephone number	53 (75.7)	32 (76.2)	39 (67.2)	22 (78.6)	36 (85.7)	0.320
E-mail address	59 (84.3)	35 (83.3)	43 (74.1)	25 (85.7)	38 (90.4)	0.211
Mailing address	41 (58.6)	25 (59.5)	28 (48.3)	20 (71.4)	34 (81.0)	0.014
Director name	65 (92.9)	37 (88.1)	49 (84.5)	24 (85.7)	37 (88.0)	0.660
Hospital affiliation	70 (100)	42 (100)	58 (100)	24 (85.7)	41 (97.6)	< 0.001
No. of positions	42 (60.0)	25 (59.5)	36 (62.1)	17 (60.7)	24 (57.1)	0.992
Names of current fellows	45 (64.3)	18 (42.9)	25 (43.1)	11 (39.3)	12 (28.6)	0.004
Names of past fellows	26 (37.1)	12 (28.6)	15 (25.9)	6 (21.4)	12 (28.6)	0.518
Past fellow's current employer	11 (15.7)	4 (9.5)	7 (12.1)	3 (10.7)	12 (28.6)	0.104
Application form	61 (87.1)	38 (90.5)	47 (81.0)	20 (71.4)	35 (83.3)	0.246
Application requirements	30 (42.9)	18 (42.9)	26 (44.8)	21 (75.0)	25 (59.5)	0.022
Application deadline	29 (41.4)	21 (50.0)	22 (37.9)	10 (35.7)	22 (52.4)	0.461
Program features
Salary	31 (44.3)	13 (31.0)	21 (36.2)	6 (21.4)	20 (47.6)	0.133
Benefits	35 (50.0)	16 (38.1)	16 (27.6)	8 (28.6)	17 (40.5)	0.089
Didactic information	58 (82.9)	36 (85.7)	49 (84.5)	22 (78.6)	33 (78.6)	0.876
Journal club	34 (48.6)	18 (42.9)	22 (37.9)	5 (17.9)	24 (57.1)	0.016
Rotation schedule	39 (55.7)	17 (40.5)	24 (41.4)	10 (35.7)	17 (40.5)	0.266
Clinic/call responsibilities	57 (81.4)	34 (81.0)	49 (84.5)	21 (75.0)	40 (95.2)	0.185
Research requirements	64 (91.4)	37 (88.1)	53 (91.4)	21 (75.0)	36 (85.7)	0.195
Conferences/travel funding	45 (64.3)	21 (50.0)	29 (50.0)	13 (46.4)	26 (61.9)	0.283
Case diversity	57 (81.4)	36 (85.7)	47 (81.0)	14 (50.0)	37 (88.1)	0.001
Surgical statistics	28 (40.0)	16 (38.1)	12 (20.7)	3 (10.7)	15 (35.7)	0.015
Social life
Community information	26 (37.1)	17 (40.5)	12 (20.6)	6 (21.4)	9 (19.0)	0.066
Wellness programs	9 (12.9)	5 (11.9)	4 (6.9)	5 (17.9)	4 (9.5)	0.618
Social media link	22 (31.4)	17 (40.5)	14 (24.1)	8 (28.6)	11 (26.2)	0.473

^a The p-values were calculated using chi-square tests. For program description and hospital affiliations, Fisher exact tests were used. Bold text indicates a p-value significant at 5% level of significance.

Discussion

Although 90.2% of AUPO-compliant ophthalmology fellowship programs have Web sites, this study identified high heterogeneity among criteria completeness. On average, the program Web sites contained only 57.2% of the 26 criteria examined; among which only 23.5% were social life criteria, compared with 63.8 and 58.4% of program demographic and program feature criteria, respectively. This suggests that some information desired by applicants might not be readily accessible, and the completeness of these Web sites may potentially be improved.

Among the criteria clusters analyzed, social life was the one that lacked information most often. Mari et al have demonstrated that wellness interventions (e.g., art therapy, exercise classes) have the ability to reduce burnout among residents.[20] Therefore, by increasing FPW content available regarding community information and wellness programs, programs demonstrate concern regarding mental health. Additionally, medical graduates often utilize the internet to seek information regarding programs.[2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [21] [22] [23] [24] [25] [26] [27] [28] By increasing their social media presence, programs can display information efficiently and better attract prospective applicants.

Although program demographics (63.8%) and program features (58.4%) were much more complete than social life (23.5%), they still present an opportunity for fellowship Web sites to improve. Surgical volume and case diversity are important factors affecting resident applicants' choice of program.[17] Additionally, the desire to acquire special skills was the most important factor affecting career choice for graduating ophthalmology residents.[16] Although 79.6% of Web sites had information on case diversity, only 30.8% of Web sites contained surgical statistics that may influence program choice. Increasing information available regarding total surgical volume and surgical diversity may attract prospective applicants into applying to specific programs.

Significant heterogeneity was observed among FPWs by subspecialty. Notably, neuro-ophthalmology program Web sites tended to have significantly fewer total (50.8%) and program feature (49.2%) criteria than other subspecialties. This might derive from the underrepresentation of certain factors on neuro-ophthalmology FPWs that are inherent to that subspecialty. For example, case diversity and surgical statistics were significantly lower among neuro-ophthalmology FPWs, which may be because neuro-ophthalmology is largely nonsurgical. However, while assessing Web site criteria, it has been noted that most neuro-ophthalmology FPWs were listed online under “neurology” fellowships rather than “ophthalmology” fellowship pages, indicating that both neurology and ophthalmology residents can apply for this fellowship. Since ophthalmology is regarded as one of the most competitive specialties by medical students, this may explain the relative thoroughness of FPWs for strictly ophthalmology fellowships.[29] However, neuro-ophthalmology FPWs more often listed application requirements. As there are different residency routes for individuals considering a career in neuro-ophthalmology, programs may need to display application requirements more clearly to avoid any confusion for ophthalmology and neurology residents.

Another criterion whose presence varied between FPWs for different subspecialties was names of current fellows, with surgical retina and vitreous FPWs showcasing trainees more often. This may be explained by differences in the number of positions available within each subspecialty. Surgical retina fellowships have 142 positions available among all AUPO-compliant programs, which is noticeably more than other subspecialties (e.g., 42 for neuro-ophthalmology and 106 for glaucoma). With an increased number of positions and the competitiveness of surgical retina fellowships, programs may be more likely to showcase their fellows for potential applicants to reach out and appear more attractive.

This study is not without limitations. First, it did not assess content on all the ophthalmology fellowships. AUPO does not set standards for ophthalmic plastic and reconstructive surgery, and other fellowship programs, such as ocular oncology, have a limited number of AUPO-compliant programs (<5). Additionally, information found on the fellowship Web sites may not be completely accurate. Web sites may have been scored as containing/not containing the information when the Web site content was outdated, or simply the feature was not listed. There was no secondary source of information to validate the accuracy of the Web sites assessed in this study. Finally, this study was not able to offer commentary on the link between content and recruitment outcomes. Studies in other fellowship matches and residency matches in other disciplines have established this link, leaving the possibility the same is true for ophthalmology.[25] [26] [27] [28] Further work may establish this to be true for ophthalmology fellowship Web sites.

Conclusion

Although most fellowship programs had a Web site, areas of content improvement in the different ophthalmology fellowship programs were identified. Within these five ophthalmology subspecialties, significant variability in program Web site content was noted, with neuro-ophthalmology programs presenting the lowest percentage of criteria. Heterogeneity was seen in different Web site elements including names of current fellows and surgical statistics. Areas of FPW content deficiencies were explored among the subspecialties, with social life criteria being the most lacking across all five of them. Addressing missing information on ophthalmology FPWs may better assist applicants in identifying programs to apply to, help optimize program-applicant fit, and reduce any unintended effect on applicants' choice of subspecialty.

Conflict of Interest

R.P.S. reports personal fees from Genentech/Roche, personal fees from Alcon/Novartis, grants from Apellis and Graybug, personal fees from Zeiss, personal fees from Bausch + Lomb, personal fees from Regeneron Pharmaceuticals, Inc. J.M.G. reports personal fees from Dompe. All other authors report no disclosures.

• References

• 1 Tang OY, Ruddell JH, Hilliard RW, Schiffman FJ, Daniels AH. Improving the online presence of residency programs to ameliorate COVID-19's impact on residency applications. Postgrad Med 2021; 133 (04) 404-408
• 2 Rockney D, Benson CA, Blackburn BG. et al. Virtual recruitment is here to stay: a survey of ID fellowship program directors and matched applicants regarding their 2020 virtual recruitment experiences. Open Forum Infect Dis 2021; 8 (08) b383
• 3 Dossani R, Karsy M, Waqas M, Menger R, Guthikonda B. A survey of applicant views regarding the neurosurgical fellowship application process. World Neurosurg 2020; 139: e373-e382
• 4 Hansberry DR, Bornstein J, Agarwal N, McClure KE, Deshmukh SP, Long S. An assessment of radiology residency program websites. J Am Coll Radiol 2018; 15 (04) 663-666
• 5 Slanetz PJ, Cooke E, Jambhekar K, Chong A. Branding your radiology residency and fellowship programs in the COVID-19 era. J Am Coll Radiol 2020; 17 (12) 1673-1675
• 6 Wang KY, Babu J, Zhang B, Jami M, Musharbash F, LaPorte D. Effect of the COVID-19 pandemic on the orthopaedic surgery residency application process: what can we learn?. J Am Acad Orthop Surg Glob Res Rev 2021; 5 (10) e21
• 7 Everett AS, Strickler S, Marcrom SR, McDonald AM. Students' perspectives and concerns for the 2020 to 2021 radiation oncology interview season. Adv Radiat Oncol 2021; 6 (01) 100554
• 8 Winters RC, Hendey GW. Do web sites catch residency applicants?. Acad Emerg Med 1999; 6 (09) 968-972
• 9 Deloney LA, Perrot LJ, Lensing SY, Jambhekar K. Radiology resident recruitment: a study of the impact of web-based information and interview day activities. Acad Radiol 2014; 21 (07) 931-937
• 10 Embi PJ, Desai S, Cooney TG. Use and utility of web-based residency program information: a survey of residency applicants. J Med Internet Res 2003; 5 (03) e22
• 11 Chu LF, Young CA, Zamora AK. et al. Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs. Anesth Analg 2011; 112 (02) 430-439
• 12 Bernstein SA, Gu A, Bernstein SL, Wei C, Vogel AC, Gold JA. Child and adolescent psychiatry fellowship program website content and accessibility. Acad Psychiatry 2021; 45 (04) 425-428
• 13 Wei C, Quan T, Wu T. et al. Assessment of the accessibility and content of dermatology fellowship websites. J Am Acad Dermatol 2021; 84 (05) 1423-1425
• 14 Ruddell JH, Tang OY, Persaud B, Eltorai AEM, Daniels AH, Ng T. Thoracic surgery program websites: bridging the content gap for improved applicant recruitment. J Thorac Cardiovasc Surg 2021; 162 (03) 724-732
• 15 Homer N, Yoon MK. Evaluation of the American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) fellowship program website content and quality. Ophthal Plast Reconstr Surg 2017; 33 (06) 471-473
• 16 Gedde SJ, Budenz DL, Haft P, Tielsch JM, Lee Y, Quigley HA. Factors influencing career choices among graduating ophthalmology residents. Ophthalmology 2005; 112 (07) 1247-1254
• 17 Yousuf SJ, Kwagyan J, Jones LS. Applicants' choice of an ophthalmology residency program. Ophthalmology 2013; 120 (02) 423-427
• 18 Programs in Compliance, Association of University Professors of Ophthalmology Fellowship Compliance Committee. Accessed April 5, 2022 at: https://aupofcc.org/programs-in-complianceinstitution=&state=All&page=1
• 19 Liu C, White RW, Dumais S. Understanding web browsing behaviors through Weibull analysis of dwell time. In: SIGIR '10: Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2010: 379-386
• 20 Mari S, Meyen R, Kim B. Resident-led organizational initiatives to reduce burnout and improve wellness. BMC Med Educ 2019; 19 (01) 437
• 21 Cohen SA, Shea K, Imrie M. An update on the accessibility and quality of online information for pediatric orthopaedic surgery fellowships. Cureus 2021; 13 (09) e17802
• 22 Yan Q, Jensen K, Field A. et al. Critical evaluation of the efficiency of colorectal fellowship websites: cross-sectional study. JMIR Med Educ 2021; 7 (04) e30736
• 23 Mukhdomi T, Woolley P, Mukhdomi J. et al. An evaluation of the online quality of content of United States Accredited Pain Medicine Fellowship Training Programs. Anesth Pain Med 2021; 11 (03) e115981
• 24 Peyser A, Abittan B, Mullin C, Goldman RH. A content and quality evaluation of ACGME-accredited reproductive endocrinology and infertility fellowship program webpages. J Assist Reprod Genet 2021; 38 (04) 895-899
• 25 Hu J, Zhen M, Olteanu C, Avram R. An evaluation of accessibility and content of microsurgery fellowship websites. Plast Surg (Oakv) 2016; 24 (03) 187-190
• 26 Silvestre J, Guzman JZ, Abbatematteo JM, Chang B, Levin LS. Evaluation of content and accessibility of hand fellowship websites. Hand (N Y) 2015; 10 (03) 516-521
• 27 Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs' web sites important to student applicants. Acad Emerg Med 2005; 12 (01) 89-92
• 28 Mahler SA, Wagner MJ, Church A, Sokolosky M, Cline DM. Importance of residency program web sites to emergency medicine applicants. J Emerg Med 2009; 36 (01) 83-88
• 29 Creed PA, Searle J, Rogers ME. Medical specialty prestige and lifestyle preferences for medical students. Soc Sci Med 2010; 71 (06) 1084-1088

Publication History

Received: 22 October 2021

Accepted: 11 January 2022

Article published online:
01 June 2022