Assessment of the Web Site Availability and Content of Pediatric Ophthalmology Fellowship Programs: A Cross-Sectional Nationwide Study

Abstract

Background Fellowship program Web sites are a crucial source of information for prospective pediatric ophthalmology applicants, especially in light of the restrictions on in-person interactions due to the coronavirus disease 2019 pandemic.

Objective This study examined all pediatric ophthalmology fellowship Web sites for availability and presented recruitment and training content.

Methods A full list of all pediatric ophthalmology and strabismus (POAS) fellowship programs were compiled from the Association of University Professors of Ophthalmology (AUPO) directory and the San Francisco Match (SFMatch) application listing. Each fellowship was queried by Google Internet search to identify the program's corresponding Web site. The content of Web sites was evaluated using 20 predetermined recruitment and training criteria. All data were collected in February 2022. Kruskal–Wallis and Mann–Whitney U tests were constructed to examine differences by geographic location, program size, number of teaching faculty, and affiliation with a top ophthalmology residency program.

Results There were 45 pediatric ophthalmology fellowships identified from the AUPO and SFMatch. All pediatric ophthalmology fellowships had an available program Web site. However, there was significant variability in content, and the average Web site had approximately half of the evaluated criteria. POAS Web sites reported a greater number of recruitment criteria than program training criteria. There were no differences by program size, number of faculty, affiliation with a top residency program, or geographic location in the number of observed criteria.

Conclusions This study provides data to drive efforts to improve pediatric ophthalmology fellowship Web sites. Pediatric ophthalmology fellowship Web sites are missing important information, and there are opportunities to improve program Web sites, regardless of program size, number of faculty, affiliation with a top residency, and geographic location. Informative, accurate Web sites may attract qualified candidates, and as a result, elevate the likelihood that interviewed candidates are well-suited to the program. As such, improvements to fellowship Web sites can streamline the application process and increase a program's success rate in the match.

Keywords

ophthalmology - pediatric ophthalmology and strabismus - fellowship - medical education - online availability - coronavirus disease 2019 pandemic

There is a shortage of pediatric ophthalmologists in the United States.[1] Yet, approximately one-third of pediatric ophthalmology and strabismus (POAS) fellowship positions go unfilled each year.[2] Fellowship program Web sites are the primary source of information used by applicants.[3] [4] Difficulty obtaining accurate information to make informed decisions on where to apply may increase the number of applications per candidate and deter well-suited applicants.[5] Restrictions on in-person interactions due to the coronavirus disease 2019 pandemic further highlight the importance of available fellowship information.[6]

A previous study of glaucoma fellowship programs found that program Web sites were consistently missing information that applicants considered important.[7] However, data are limited regarding the availability and completeness of information from POAS fellowship Web sites. This study examined the content on POAS fellowship Web sites to understand what information is currently available online.

Methods

We analyzed the availability and content of all 45 POAS fellowship programs, which were identified from the Association of University Professors of Ophthalmology (AUPO) directory of programs in compliance (https://aupofcc.org/programs-in-compliance?institution=&state=All&sub_specialty%5B%5D=pediatric-ophthalmology) and the San Francisco Match (SFMatch) application listing. Program name, affiliated institution, program size, and geographic location were abstracted from the AUPO and SFMatch Web sites, respectively. We then assessed the availability and content of each program's Web site.

We first designed a standardized search strategy to identify POAS program Web sites prior to the start of data collection. Data were collected by two independent researchers, with consensus meeting to resolve any varying assessments. To determine the availability of Web sites, each POAS fellowship was searched on Google by: “program name” + “affiliated institution” + “pediatric ophthalmology fellowship.” A POAS fellowship program was considered available if the Web site could be identified from the Google search results. If no Web site could be found by Google Internet search, “pediatric ophthalmology fellowship” was queried using the search function of the affiliated parent institution's Web site. If no Web site could be identified from either Google Internet search or search of the affiliated institution's Web site, then that program was determined to lack an available program Web site. Though we decided on this two-step search strategy a priori, all Web sites could be identified by Google search, and the second step of the search was not used.

The content of Web sites was evaluated for 10 recruitment and 10 program training factors ([Table 1]). These criteria were selected based on AUPO certification criteria, similar studies, and surveys of what content is important to applicants.[7] [8] Recruitment factors were program description, list of teaching faculty, program contact information, link to application, current fellows, prior fellows, alumni placement, demographic characteristics of patient community, information about program's location, and salary and benefits. Program training factors were overview of fellowship curriculum and schedule, affiliated hospitals and clinics, surgical experience, research requirement and opportunities, opportunities to present research at conferences and meetings, resident and student teaching responsibilities, call schedule, grand rounds, journal club, and fellow evaluation process. All Web site materials, including video content, were reviewed to determine the presence or absence of each criterion.

Table 1 Proportion of fellowship program Web sites reporting each recruitment and training criteria

Recruitment criteria		Training criteria
Criterion	Frequency (proportion)	Criterion	Frequency (proportion)
Program description	44 (97.8%)	Rotation overview	45 (100.0%)
Listing of teaching faculty	30 (66.7%)	Affiliated hospitals and sites	44 (97.8%)
Program contact name/email	28 (62.2%)	Surgical experience	41 (91.1%)
Link to application	25 (55.6%)	Research requirement and opportunities	39 (86.7%)
Salary and benefits	22 (48.9%)	Resident teaching role	32 (71.1%)
Current fellow names	13 (28.9%)	Journal club	29 (64.4%)
Information about city	12 (26.7%)	Academic conferences	26 (57.8%)
Prior fellow names	10 (22.2%)	Grand rounds	24 (53.3%)
Alumni placement	6 (13.3%)	Call schedule	19 (42.2%)
Information on patient demographic	2 (4.4%)	Fellow evaluation process	7 (15.6%)

Summation of the 20 criteria calculated a total score of each Web site's content (range: 0–20). Mean and standard deviation were estimated. We also examined differences in fellowship Web site content, stratifying by geographic location (Northeast/Midwest/South/West US, Canada) number of fellows (1/2+ fellows) and teaching faculty (≤8/≥9 faculty), and affiliation with a top-ranked ophthalmology residency program. A top ophthalmology residency program was defined as those ranked in the 2021 Top 12 Best Residency programs by the Ophthalmology Times.[9] Kruskal–Wallis and Mann–Whitney U tests were used to assess the association with Web site content.

All data were collected from program Web sites over a 2-day period in July 2022. Data maintenance and analyses were conducted in SAS version 9.4 (SAS Institute, Cary, NC). A two-sided p-value less than 0.05 was considered statistically significant. The Northwestern University Institutional Review Board determined this study was not human subjects research because all data collected were publicly available online and the results are presented in aggregate.

Results

Out of 45 identified POAS fellowships, all 45 programs had a Web site to provide fellowship information identifiable from a Google Internet query. These programs included 22 POAS fellowship programs with one fellow (48.9%) and 23 programs with more than or equal to two fellows (51.1%). The greatest number of programs were located in the Midwestern United States (n = 19, 42.2%), followed by South (n = 10, 22.2%), West (n = 8, 17.8%), and Northeast regions (n = 5, 11.1%). We also identified and included three programs located in Canada.

The mean number of total criteria on POAS fellowship Web sites was 11.1 (standard deviation 3.3, range: 3–18), which included a mean of 4.3 recruitment criteria (standard deviation 1.9, range: 1–9) and 6.8 training criteria (standard deviation 2.1, range: 2–10). POAS fellowships reported fewer recruitment criteria than training criteria (paired t-test, p < 0.0001).

The factors found most commonly on fellowship program Web sites were rotation overview (100.0%), overall program description (97.8%), rotation overview (97.7%), and affiliated hospitals and sites (97.8%), whereas patient demographic information (4.4%) and alumni placement (13.3%) were reported least frequently ([Table 1]). Notably, 5 out of 10 training criteria were reported on more than 70% of POAS Web sites, whereas only one recruitment criterion was reported on more than 70% of Web sites.

There were no differences in number of overall criteria on Web sites by number of fellows, number of faculty, geographic location, or affiliation with a top ophthalmology residency (p ≥ 0.36) ([Table 2]). Similarly, we found no differences in the number of recruitment or training criteria reported by the number of fellows, number of faculty, geographic location of the program, or affiliation with a top residency program (p ≥ 0.09).

Table 2 Association of program size and geographic location with Web site reporting of criteria

Classification	Recruitment score		Training score		Total score
	Mean (SD)	p-Value	Mean (SD)	p-Value	Mean (SD)	p-Value
Number of fellows
1 fellow	4.0 (1.8)	0.45	7.3 (2.3)	0.09	11.2 (3.1)	0.44
≥2 fellows	4.6 (2.1)		6.3 (1.9)		10.9 (3.5)
Number of faculty
1–8 faculty	4.5 (2.0)	0.16	7.2 (1.6)	0.69	11.7 (2.8)	0.36
≥9 faculty	5.6 (1.7)		7.2 (2.4)		12.8 (3.5)
Geographic region
Midwest	4.3 (1.9)	0.36	6.9 (2.1)	0.33	11.3 (3.1)	0.43
Northeast	5.0 (2.5)		7.4 (1.5)		12.4 (3.5)
South	4.0 (1.9)		5.6 (2.2)		9.6 (3.8)
West	4.8 (1.8)		7.3 (2.5)		12.0 (3.1)
Canada	2.3 (1.5)		7.7 (1.5)		10.0 (2.0)
Affiliated with top 12 residency program
No	4.3 (1.8)	0.72	6.8 (2.2)	0.81	11.1 (3.2)	0.70
Yes	4.3 (2.4)		6.7 (1.9)		11.0 (3.8)

Abbreviation: SD, standard deviation.

Discussion

This cross-sectional nationwide review of POAS fellowships found all 45 programs have a Web site available by Google search. However, there was significant variation in the content posted on these Web sites. The average Web site contained approximately half the evaluated criteria, missing multiple pieces of information that applicants considered important.[7] [8] There were no differences in the Web site content scores by program size, affiliated institution ranking, or geographic location. Taken together, these findings suggest there are significant opportunities to improve the content of POAS program Web sites, regardless of program size, affiliation, and geographic location. Medical educators and administrators may consider working together to improve information availability on program Web sites.

All POAS fellowships had an available Web site easily discovered by a Google search of the Internet. The importance of having a functional Web site has been well-established in literature for program recruitment. Orthopaedic surgery fellowship applicants used the program Web site more often than any other online resource.[3] Of applicants to emergency medicine residency, 96% used the Internet to learn more about programs, and 40 and 47% reported the program Web site was very important and moderately important, respectively.[10] Similarly, 98% of applicants to the anesthesia residency at Stanford University used the program Web site to learn about the program, and 56% used the Web site to determine whether to apply to the program.[11] Previous studies found most emergency medicine residency applicants use the Web site content to determine whether to apply to the program, and almost half did not apply to a program because of the Web site.[12] However, these studies did not specifically survey prospective POAS fellows. Further research is needed to validate the importance of program Web sites specifically among POAS candidates. Nonetheless, POAS candidates likely benefit from available, informative program Web sites when determining applications and rank list.

Fellowship program Web sites help applicants discern which programs to apply to and create their rank list.[10] Inconsistent online information may contribute to candidates submitting applications to more programs. A survey of surgery fellowship applicants found 62% would have removed programs from their list if the program's Web site had additional information.[5] This results in higher applicant spending on application fees and greater time invested by faculty to review applications and conduct interviews. Multiple solutions have been proposed to address inefficiencies, including limiting the number of applications and interviews and implementing tiered rank lists.[13] Moreover, without reliable information on Web sites, applicants may rely on anecdotal sources, such as blogs and forums, to assess program fit.

Additionally, easily accessible information may increase a program's matching success. Informative Web sites help candidates determine their suitability before submitting an application, so programs are more likely to interview a cohort of applicants who understand their fit with the program. Conversely, it is less likely a candidate will realize post-interview that the program was a poor fit and rank the program poorly.[5] Studies are needed to better understand the association of fellowship Web sites with program matching success and drive quality improvement efforts. Low match rates are not limited to pediatric ophthalmology fellowships. Comparable fellowship match rates were observed in all other pediatric surgical subspecialties, except for pediatric general surgery.[14] Similar studies may be warranted to understand the online presence of fellowship programs in other pediatric surgical subspecialties and optimize program match rates.

Notably, POAS fellowship Web sites listed fewer recruitment than training criteria, highlighting that this could be an area for programs to target updating their Web sites. Of the recruitment criteria examined in this study, less than 1 in 4 programs provided information on current fellows, prior fellows, alumni placement, and information about the city and patient demographic. Only six POAS fellowships reported alumni placement, even though prospective applicants likely consider this important information.[15] Alumni placement offers insight into the program's track record of placement by region and practice type (i.e., academic institution, private practice). This helps applicants know whether the program's goals align with their desired career path. A similar analysis of American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) fellowship Web sites found none of the 53 ASOPRS programs reported information about alumni job placement.[16] Profiles of current and prior fellows can also help applicants better understand the background of a fellow who will likely thrive in the program. Furthermore, information about the city and patient demographic may allow prospective fellows to have a more accurate impression of the surrounding community before applying or interviewing. These are key areas that POAS fellowship Web sites can improve availability of information prior to the interview.

We evaluated fellowship program Web sites using 10 recruitment and 10 training criteria based on previous studies.[7] [17] This allowed for comparison to analogous assessments of glaucoma and surgical retina subspecialties. Content scores of POAS Web sites in this study were overall comparable to those found in previous assessments of glaucoma and surgical retina fellowship Web sites.[7] [17] However, there are a few notable differences. Teaching faculty were listed on 66.7% of POAS Web sites, compared with 87.7% of glaucoma and 88.2% of surgical retina Web sites. Also, resident teaching responsibilities were reported on 71.1% of POAS Web sites, compared with 82.5% of glaucoma and 76.5% of surgical retina program Web sites. Lastly, program contact information could be identified on 62.2% of POAS Web sites, which was similar to 68.4% of glaucoma Web sites, and lower compared with 94.1% of surgical retina Web sites. POAS fellowships may consider including a list of teaching faculty, resident teaching responsibilities, and program contact information to facilitate information gathering for prospective POAS fellows.

Interestingly, there were no differences in fellowship Web site content by geographic region, ranking of affiliated institution, or program size, consistent with a previous study that examined program Web sites of glaucoma fellowships.[7] These findings suggest there are opportunities for improvement in Web site transparency across programs, regardless of geography and size.

Strengths of this study include being a nationwide survey of all POAS fellowship programs in the United States. However, there are also some limitations that merit mention. First, there may be criteria that are important to applicants not included in our evaluation. We selected our 20 study criteria based on previous studies, which allowed for comparison and benchmarking with other subspecialties.[7] [8] However, the selection of study criteria based on what is important to non-POAS means there may be other factors that are uniquely important to POAS fellowships and are not captured here. No previous studies have surveyed prospective POAS fellows to identify the most important criteria for them in particular. Future studies are needed to validate best-practice Web site criteria for POAS fellows. Second, program Web sites are dynamic and may have changed since our evaluation. To reduce bias, all programs were evaluated within a 2-day period in July 2022. Third, the cross-sectional nature of the study design precluded any conclusions of trends over time.

Conclusions

In conclusion, all POAS fellowships have an available Web site, yet there are opportunities for programs to improve the content on their Web sites. The mean POAS program Web site contained approximately half of the evaluated content criteria, with lower recruitment subscores than training subscores. Program size and location were not associated with differences in Web site content score, suggesting that these opportunities exist across programs. Informative program Web sites can attract qualified candidates well-suited to that program, whereas Web sites lacking reliable information may deter qualified candidates. As such, improvements have the potential to increase a program's matching success within the existing application process.

Conflicts of Interest

None.

Authors' Contributions

BT Cheng had full access to all the data in the study and takes responsibility for the integrity of the data and accuracy of the data analysis. AB Kim, BT Cheng, and SP Kurup conceptualized and designed the study. AB Kim, KS Chan, and BT Cheng helped in acquisition of data. AB Kim, KS Chan, BT Cheng, and SP Kurup contributed to analysis and interpretation of data. AB Kim, BT Cheng, and SP Kurup drafted the manuscript. AB Kim, KS Chan, BT Cheng, and SP Kurup helped in critical revision of the manuscript for important intellectual content. BT Cheng and AB Kim contributed to statistical analysis.

• References

• 1 Estes R, Estes D, West C, Zobal-Ratner J, Droster P, Simon J. The American Association for Pediatric Ophthalmology and Strabismus workforce distribution project. J AAPOS 2007; 11 (04) 325-329
• 2 Dotan G, Karr DJ, Levin AV. Pediatric ophthalmology and strabismus fellowship Match outcomes, 2000-2015. J AAPOS 2017; 21 (03) 181.e1-181.e8
• 3 Shaath MK, Avilucea FR, Lim PK, Warner SJ, Achor TS. Increasing fellow recruitment: how can fellowship program websites be optimized?. J Am Acad Orthop Surg 2020; 28 (24) e1105-e1110
• 4 Shaath MK, Yeranosian MG, Ippolito JA, Adams MR, Sirkin MS, Reilly MC. Evaluation of the content and accessibility of web sites for accredited orthopaedic trauma surgery fellowships. J Bone Joint Surg Am 2018; 100 (09) e60
• 5 Watson SL, Hollis RH, Oladeji L, Xu S, Porterfield JR, Ponce BA. The burden of the fellowship interview process on general surgery residents and programs. J Surg Educ 2017; 74 (01) 167-172
• 6 Czawlytko C, Smith E, Awan O, Resnik C, Hossain R. The effect of virtual interviews and social media on applicant decision-making during the 2020–2021 resident match cycle. Acad Radiol 2021
• 7 Cohen SA, Fisher AC, Pershing S. Evaluation of Website accessibility and content for all Glaucoma Fellowship Programs in the United States. Ophthalmol Glaucoma 2021
• 8 Embi PJ, Desai S, Cooney TG. Use and utility of Web-based residency program information: a survey of residency applicants. J Med Internet Res 2003; 5 (03) e22
• 9 Charters L. 2021 Best Program results: Wilmer Eye Institute grabs top spot. 2021 Accessed February 22, 2023 at: https://www.ophthalmologytimes.com/view/2021-best-program-results-wilmer-eye-institute-grabs-top-spot
• 10 Mahler SA, Wagner MJ, Church A, Sokolosky M, Cline DM. Importance of residency program web sites to emergency medicine applicants. J Emerg Med 2009; 36 (01) 83-88
• 11 Chu LF, Young CA, Zamora AK. et al. Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs. Anesth Analg 2011; 112 (02) 430-439
• 12 Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs' web sites important to student applicants. Acad Emerg Med 2005; 12 (01) 89-92
• 13 Melcher ML, Ashlagi I, Wapnir I. Matching for fellowship interviews. JAMA 2018; 320 (16) 1639-1640
• 14 Oke I, Heidary G, Mantagos IS, Shah AS, Hunter DG. Comparison of fellowship match opportunities and results across pediatric surgical subspecialities. J AAPOS 2022; 26 (03) 145-148
• 15 Lu F, Vijayasarathi A, Murray N, Hamid S, Khosa F. Evaluation of pediatric radiology fellowship website content in USA and Canada. Curr Probl Diagn Radiol 2021; 50 (02) 151-155
• 16 Homer N, Yoon MK. Evaluation of the American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) fellowship program website content and quality. Ophthal Plast Reconstr Surg 2017; 33 (06) 471-473
• 17 Cohen SA, Pershing S. Accessibility and content of vitreoretinal surgery fellowship program websites. Ophthalmol Retina 2022; 6 (05) 434-436

Publication History

Received: 16 May 2022

Accepted: 27 September 2022

Article published online:
14 March 2023