National Access to EyeSi Simulation: A Comparative Study Among U.S. Ophthalmology Residency Programs

Abstract

Purpose The aim of this study was to evaluate regional disparities in access to EyeSi surgical simulation training among U.S. ophthalmology residency programs.

Methods Access to EyeSi simulation was determined from sales data (2021) provided by VRMagic. Key demographic metrics of the primary counties of U.S. residency training programs were retrieved from the U.S. Census Bureau Database (2019) and PolicyMap (2021). Demographic metrics, Veterans Affairs (VA) hospital affiliation, and Doximity residency program ranking (2021) were compared using the Mann–Whitney U test and Fisher's exact test.

Results A total of 124 residency training programs across 95 U.S. counties were included. Regional density (number of EyeSi simulators/million people) was calculated; the west had a significantly lower density when compared with the northeast (NE), south, and midwest (0.4 vs 1.0, 1.3, 1.1, respectively). In the NE, there was a significantly lower population of Blacks (p = 0.01), Hispanics (p = 0.028), and Native Americans (p = 0.008) residing in counties with access to EyeSi, compared with counties without EyeSi access. Programs with EyeSi access (N = 95) had a median Doximity ranking of 52.5, whereas programs without EyeSi access (N = 35) had a lower median ranking of 94 (p < 0.001).

Conclusion Our analysis demonstrates significant disparities in access to EyeSi simulation training in the United States that could disproportionately impact minority communities. Access to an EyeSi simulator was associated with higher residency ranking independent of VA affiliation.

Keywords

EyeSi simulation - distribution - ophthalmology residency programs - program ranking - VA affiliation

The EyeSi (VRMagic, Haag-Streit, Switzerland) is a commercially available surgical simulation system designed primarily to train novice ophthalmic surgeons in the skills and maneuvers required to perform cataract and vitreoretinal surgery. Training has led to increased productivity, improved surgical outcomes, and reduced cost to the health care system.[1] [2] An additional study has even demonstrated a high correlation between performance on a virtual reality simulator and actual cataract surgery.[3] In a large retrospective analysis of posterior capsular rupture among novice surgeons, introduction of EyeSi training was associated with a 38% reduction in complication rate.[4] Similar studies in laparoscopic and endovascular simulation have demonstrated transferability to actual operating room skills.[5] [6] While commercially available synthetic tissue simulators exist, challenges in accurately measuring cumulative objective performance limit the ability to assess efficacy and transferability of acquired skills.

Given the demonstrable positive impact that incorporation of EyeSi training has had for many trainees, the majority of residency programs in the United States have acquired a unit through private or public funding, including federal grants for support of Veterans Affairs (VA) hospitals. As acquisition, software licensing, and lifetime maintenance of the EyeSi can often exceed US$300,000, without the potential for revenue generation, many academic centers struggle acquiring and maintaining an EyeSi unit. Our analysis was designed to characterize and systematically compare the demographics of patients in the immediate vicinity of residency programs with and without an EyeSi training unit. Given that EyeSi training has been demonstrated to reduce complications by novice surgeons, our aim was to analyze whether an asymmetric distribution of simulator training units potentially impacts underresourced communities and vulnerable populations at risk of poor health outcomes.[7] [8] [9] We also analyzed the association between access to an EyeSi unit and VA affiliation on residency ranking by Doximity Residency Navigator (www.doximity.com, San Francisco, CA). To our knowledge, this is the first study reporting the national distribution of EyeSi simulators among U.S. ophthalmology residency programs.

Methods

Data Collection

A comprehensive list of accredited U.S. ophthalmology residency programs, along with location (county) and presence or absence of a VA affiliation, was compiled from both 2021 San Francisco Match and Doximity at the time of the analysis (2021). The EyeSi training sites were compiled from a comprehensive database of U.S. sales provided by the sole manufacturer of the EyeSi simulator (VR Magic, Haag-Streit, Switzerland) and cross referenced to the associated training program using Accreditation Council for Graduate Medical Education (ACGME) and Doximity residency navigator databases. The 2019 American Community Survey (ACS), which is a component of the U.S. Census data, was used to provide county-level population statistics such as age (≥ 65 years old), poverty and unemployment rates, race (White, African American, American Indian, Asian, Hispanic or Latino, other race), and health insurance coverage data from Medicaid and Medicare. When programs had access to multiple EyeSi units, we considered each affiliated hospital with EyeSi as a distinct training site in the analysis.

We analyzed a total of 124 ophthalmology residency programs (including all accredited U.S. ophthalmology programs as listed on Doximity) across 95 counties in the United States and counted each EyeSi training unit within a program. We used “residency hospitals” as a surrogate for ophthalmology residency programs, which enabled us to account for each EyeSi within a program. There were a total of 130 hospitals included in this analysis; each residency hospital was affiliated with an ophthalmology residency program, while some programs were affiliated with multiple hospitals with EyeSi units.

Hospitals with an EyeSi unit and those without an EyeSi unit were stratified based on region and subregion. Regional analysis (excluding Puerto Rico) was performed to assess the distribution of EyeSi units across the four regions of the mainland United States—northeast (NE), south (S), midwest (MW), and west (W)—and identify any demographic or socioeconomic differences among the counties within each region. Subregional analysis was performed to examine the distribution of EyeSi units within New York City (NYC) and Long Island. We geographically mapped the distribution of EyeSi units among ophthalmology residency programs in the United States using PolicyMap.

In addition, data on ophthalmology program ranking and VA affiliation were retrieved from Doximity.[10] We evaluated residency program reputation rankings and VA affiliation in programs with an EyeSi unit and in those without an EyeSi unit.

Statistical Analysis

Among the U.S. counties with ophthalmology residency programs, county-level demographic and socioeconomic characteristics were summarized by count (%) for categorical variables and median (interquartile range) for continuous variables, according to the presence or absence of the EyeSi unit, and compared using Fisher's exact test for categorical variables and the Mann–Whitney U test for continuous variables. A subgroup analysis by U.S. regions (NE, W, S, MW) and within NYC was performed. In addition, we calculated the density of EyeSi training units per million people by dividing the number of units by total population size (in millions) and compared the density between regions using a Poisson regression. In addition, we used a Mann–Whitney U test to examine the association between Doximity ranking with EyeSi unit presence and VA affiliation.

Results

EyeSi Distribution in the United States

The proportion of “Programs with an EyeSi/Total number of programs” by region was found to be as follows: NE, 0.6 (21/35); S, 0.8 (40/50); MW, 0.79 (22/28); W, 0.75 (12/16), with no statistically significant difference among the regions (p = 0.21; [Table 1]; [Fig. 1]). In consideration of population size, the overall number of EyeSi training units in the United States per million people was 0.9. We observed a disproportionate access to EyeSi training units per million people: NE, 1.0 EyeSi units/million people; S, 1.3; MW, 1.1; W, 0.4 (p = 0.001), with the W region having the lowest number of EyeSi training units per million people.

Table 1 Regional analysis of EyeSi in the NE, S, MW, and W

	Total	NE	S	MW	W	p-Value (overall)
Hospitals	129	35	50	28	16
Counties[a]	94	22	39	20	13
EyeSi, n	95	21	40	22	12	0.21[b]
EyeSi per 1 million people	0.9	1	1.3	1.1	0.4	0.001[c]
EyeSi per resident	0.06	0.04	0.05	0.06	0.05	0.001[c]

Abbreviations: MW, midwest; NE, northeast; S, south; W, west.

^a If there are multiple hospitals in the same county, the county was counted only once.

^b p-Value by the Fisher's exact test.

^c p-Value by a Poisson regression.

Fig. 1 Distribution of EyeSi across the regions of the United States.

Given the density of ophthalmology training programs, a subregional analysis of NYC and Long Island included a total of 10 hospitals with ophthalmology residency programs distributed across 5 counties ([Supplementary Table S1]; [Fig. 2]). There were a total of six EyeSi units within this subregion: four in Manhattan, one in Brooklyn, one in Long Island, 0 in the Bronx, and 0 in Queens (p = 0.076). The numbers of EyeSi training units per million people in these counties were 2.5, 0.4, 0.7, 0, and 0, respectively (p = 0.02).

Fig. 2 Comparing a map of the medically underserved areas of New York City and ophthalmology residency programs with EyeSi access.

Demographic Analysis

To identify any demographic and socioeconomic features associated with EyeSi acquisition, we compared census data of counties in which residency hospitals are located. Within the NE, a greater proportion of the U.S. population over the age of 65 years reside in counties with programs without an EyeSi unit (p = 0.033) ([Table 2]). A similar result was found for overall “insured” patients (p = 0.014), and “insured” patients over the age of 65 years (p = 0.033). There was a significant disparity between the number of people who live in counties with and without access to EyeSi units in the NE (p = 0.011).

Table 2 Distribution of EyeSi within the northeast by demographic and socioeconomic parameters

Northeast	EyeSi	No EyeSi	P-value
Residency hospitals[a]	21	14	–
Corrected population[b]	407,176	725,436	0.011[c]
Age ≥ 65 y	69,220	128,128	0.033[c]
White	230,869	410,202	0.2
African American	59,489	128,669	0.011[c]
American Indian	1,584	3,063	0.008[c]
Asian	24,077	30,568	0.105
Hispanic or Latino	62,436	140,994	0.028[c]
Insured (civilian noninstitutionalized)	388,853	687,413	0.014[c]
Insured age ≥ 65 y (civilian noninstitutionalized)	68,597	127,568	0.033[c]
Dual insurance by Medicare and Medicaid	12,861	21,120	0.099

Note: Values are median for continuous variables and count (%) for categorical variables. p-Value by Mann–Whitney U test for continuous variables and Fisher's exact test for categorical variables.

^a Hospitals with ophthalmology residency programs.

^b Population of each county was modified by dividing the number of hospitals with ophthalmology residency programs. Age ≥ 65 years and race were calculated using the corrected population size.

^c p-Values are statistically significant.

With respect to self-reported race and ethnicity, compared with Whites and Asians, there was a significantly lower population of Blacks (p = 0.01), Hispanics (p = 0.028), and Native Americans (p = 0.008) residing in the counties with access to EyeSi. However, there was no significant difference in demographic characteristics in other regions.

Impact on Program Ranking

Programs with an EyeSi unit (N = 95) had an overall median rank of 52.5, whereas programs without an EyeSi unit (N = 35) had a median rank of 94 (p < 0.001) ([Table 3]). The affiliation of residency programs with a VA hospital was independently associated with a higher ranking when compared with unaffiliated residencies (58.5 vs 69, respectively; p = 0.053) ([Table 4]). Subgroup analysis of programs with an EyeSi unit demonstrated that VA affiliation did not have a significant impact on rank (54 vs 51, respectively; p = 0.808). For programs without an EyeSi unit, presence of a VA affiliation was associated with a higher overall rank compared with programs without a VA (88.5 vs 112, respectively; p = 0.002). Conversely, subgroup analysis by VA affiliation showed that EyeSi was associated with a higher rank independent of VA affiliation (among non–VA-affiliated programs: 51 vs 112, p < 0.001; among VA-affiliated programs: 54 vs 88.5, p = 0.034).

Table 3 Comparing program rankings between the EyeSi and no EyeSi groups

	EyeSi	No EyeSi	p-Value
	Median rank	Median rank
Overall United States	52.5 (26.2–77.8)	94 (59.5–111.5)	<0.001[a]
Northeast	49 (36–81)	104.5 (74.5–112.8)	0.002[a]
Midwest	43.5 (24.5–55.8)	95 (65.2–112)	0.005[a]
South	73 (39–83)	91 (49.8–100)	0.076
West	27 (11.8–56.2)	38 (30.5–57.8)	0.52

Note: Values are median (interquartile range). p-Value by Mann–Whitney U test.

^a p-Values are statistically significant.

Table 4 Association between EyeSi access and Veterans Affairs affiliation on residency program ranking

	VA affiliation	No VA affiliation	p -Value
Overall ranking	58.5 (29.5–83.5)	69 (39–109)	0.053
Programs without EyeSi	88.5 (38.2–96.2)	112 (82–117)	0.002[a]
Programs with EyeSi	54 (26.2–77.8)	51 (28–78)	0.808
	EyeSi	No EyeSi	p -Value
Overall ranking	52.5 (26.2–77.8)	94 (59.5–111.5)	<0.001[a]
Ranking without VA	51 (28–78)	112 (82–117)	<0.001[a]
Ranking with VA	54 (26.2–77.8)	88.5 (38.2–96.2)	0.034[a]

Abbreviation: VA, Veterans Affairs.

Note: Values are median (interquartile range). p-Value by Mann–Whitney U test.

^a p-Values are statistically significant.

Discussion

EyeSi surgical simulation training can provide opportunities for advancement of surgical skills that have been demonstrated to improve clinical outcomes and productivity.[1] [2] [4] Simulation training in the virtual environment furthermore offers a more reproducible experience that is less dependent on trainer expertise compared with wet laboratory sessions that utilize ex vivo animal specimens or simulated tissue. Furthermore, the ability to generate objective metrics that are able to be compared with other students through a centralized network offers nuanced performance data. Motion tracking analysis has demonstrated a strong correlation between performance on the EyeSi and actual cataract surgery,[3] [11] [12] with the ability to even distinguish between novice, intermediate, and experienced surgeons.[13] [14]

Our study demonstrated a disproportionate access to EyeSi simulation among residents training in the United States. Regional analysis on the density of access to EyeSi training per population (number of simulators/million people) showed a significantly lower density in the west compared with other regions (p = 0.001). In addition, in the NE, there were more people who lived in counties with residency hospitals without access to EyeSi simulation (p = 0.011). There was also a disproportionate distribution in race, demographic, and socioeconomic parameters between the “EyeSi” and the “No EyeSi” groups. Specifically, the majority of the insured population ≥ 65 years, Black, Afro-Latino, and Native Americans resided in counties with residency hospitals without access to an EyeSi.

Based upon the relatively high acquisition and maintenance cost, we hypothesized that more resourced residency hospitals centered near more affluent and commercially insured patient populations may be more capable of acquiring an EyeSi unit. However, no statistically significant difference in median household income was noted when comparing the counties of programs with and without an EyeSi unit across the whole United States. We then performed a subgroup analysis of NYC training programs, where large income disparities are known to exist throughout the boroughs within a small densely populated geography. Median household income for the Bronx, Brooklyn, Queens, and Manhattan was found to be $41,432, $66,937, $73,696, and $93,651, respectively,[15] with an overall poverty rate of 17.3% relative to the U.S. average of 11.4%.[16] Concentrations of socioeconomic disparity are found in the South Bronx, East and Central Harlem, and North and Central Brooklyn. These regions are characterized by the highest number of Black and Afro-Latino residents ([Fig. 2]),[17] who disproportionately reside in medically underserved areas of NYC.[18] [19] [20] [21] The asymmetric distribution of EyeSi units in NYC affects these populations specifically, where notably four EyeSi units are placed in Manhattan, one in Brooklyn, and no EyeSi units in the Bronx or Queens. There are two ophthalmology training programs based in the Bronx County (Montefiore Medical Center and BronxCare Health System), and two programs located in Queens County (St. John's Episcopal and Jamaica Hospital). The residents in these programs do not currently have access to simulation training with the EyeSi. Some residents from programs in Manhattan rotate through the Bronx VA Hospital, which offers a simulation training opportunity, but access is limited only to those residents secondary to logistical constraints.

To evaluate the association between having an EyeSi unit and residency ranking, we examined programs using the Doximity residency navigator. Analysis of program rankings showed that residency programs with an EyeSi had on average a higher rank (median rank = 52.5) compared with programs without access to EyeSi (median rank = 94). It is important to note that residency ranking is based upon multiple factors utilizing a proprietary ranking methodology,[10] and the association between EyeSi presence and higher rank should not be interpreted as causal in nature given the multitude of confounding factors such as program size and reputation.[22]

As many residency programs in the United States are affiliated with the U.S. Department of Veterans Affairs,[23] which can provide funding for integration of an EyeSi surgical simulator, we further analyzed the impact of the presence of a VA Hospital on program rank. VA-affiliated programs had a higher median rank compared with programs without a VA (58.5 vs 69, p = 0.05).[12] [24] Among residency hospitals with an EyeSi unit, ranking was not statistically different between the VA-affiliated (54) and non–VA-affiliated programs (51) (p = 0.808). However, for residency programs without an EyeSi unit, residencies with a VA affiliation had a statistically significant higher rank (median = 88 vs 112, respectively; p = 0.002). These findings suggest that residency programs with an EyeSi tend to be higher ranked regardless of VA affiliation, the presence of which also independently positively impacts residency ranking.

Ideally, all ophthalmology residents should have access to EyeSi training, regardless of resource allocations and federal affiliation, particularly given recent literature demonstrating many benefits of incorporation of formal surgical simulation training for novice surgeons.[1] [2] [4] An important finding from Ferris et al[4] was that novice surgeons were able to achieve similar rates in posterior capsule rupture reduction regardless of whether the EyeSi was located on- or off-premises, suggesting the potential benefits of a centralized training model.

In conclusion, this analysis produced a snapshot of ophthalmic training in the United States and demonstrated an asymmetric distribution of EyeSi units across ophthalmology training programs, which can disproportionately impact underresourced communities and vulnerable populations who are already at risk of poor health outcomes. An analysis of national resource allocation and training models needs to be performed to equilibrate access to high-fidelity simulation training for all programs and communities. Further studies should examine the correlation between presence of EyeSi and resident complication rates in cataract surgery nationwide to provide additional support to improve EyeSi access for all residency programs.

Study Limitations

Data acquired from the U.S. Census Bureau has inherent flaws and associated reporting biases, which may have impacted the raw data utilized for this analysis. We examined residency programs based on their primary address and counties; however, many training programs are structured to have residents rotate through large geographies and varied health care systems, which is impossible to enumerate.

Conflict of Interest

None declared.

Acknowledgments

The authors thank the Manhattan Eye and Ear Ophthalmology Alumni Foundation, Ronald M Burde Resident Education Fund, and the Center for Ophthalmic Innovation at Montefiore for their support and the VR Magic for providing the sales data.

• References

• 1 Jamison A, Benjamin L, Lockington D. Quantifying the real-world cost saving from using surgical adjuncts to prevent complications during cataract surgery. Eye (Lond) 2018; 32 (09) 1530-1536
• 2 Zendejas B, Brydges R, Hamstra SJ, Cook DA. State of the evidence on simulation-based training for laparoscopic surgery: a systematic review. Ann Surg 2013; 257 (04) 586-593
• 3 Thomsen AS, Smith P, Subhi Y. et al. High correlation between performance on a virtual-reality simulator and real-life cataract surgery. Acta Ophthalmol 2017; 95 (03) 307-311
• 4 Ferris JD, Donachie PH, Johnston RL, Barnes B, Olaitan M, Sparrow JM. Royal College of Ophthalmologists' National Ophthalmology Database study of cataract surgery: report 6. The impact of EyeSi virtual reality training on complications rates of cataract surgery performed by first and second year trainees. Br J Ophthalmol 2020; 104 (03) 324-329
• 5 Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 2007; 246 (05) 771-779
• 6 Maertens H, Aggarwal R, Moreels N, Vermassen F, Van Herzeele I. A proficiency based stepwise endovascular curricular training (prospect) program enhances operative performance in real life: a randomized controlled trial. Eur J Vasc Endovasc Surg 2017; 54 (03) 387-396
• 7 Marmot M, Friel S, Bell R, Houweling TA, Taylor S. Commission on Social Determinants of Health. Closing the gap in a generation: health equity through action on the social determinants of health. Lancet 2008; 372 (9650): 1661-1669
• 8 Braveman P, Egerter S, Barclay C. What Shapes Health-Related Behaviors? The Role of Social Factors. Exploring the Social Determinants of Health: Issue Brief No. 1. Princeton, NJ: Robert Wood Johnson Foundation; 2011: 47
• 9 Stringhini S, Sabia S, Shipley M. et al. Association of socioeconomic position with health behaviors and mortality. JAMA 2010; 303 (12) 1159-1166
• 10 Doximity. Residency Navigator Methodology. 2019 . Accessed August 26, 2021 at: https://residency.doximity.com
• 11 McCannel CA, Reed DC, Goldman DR. Ophthalmic surgery simulator training improves resident performance of capsulorhexis in the operating room. Ophthalmology 2013; 120 (12) 2456-2461
• 12 Seymour NE, Gallagher AG, Roman SA. et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002; 236 (04) 458-463 , discussion 463–464
• 13 Spiteri AV, Aggarwal R, Kersey TL. et al. Development of a virtual reality training curriculum for phacoemulsification surgery. Eye (Lond) 2014; 28 (01) 78-84
• 14 Mahr MA, Hodge DO. Construct validity of anterior segment anti-tremor and forceps surgical simulator training modules: attending versus resident surgeon performance. J Cataract Refract Surg 2008; 34 (06) 980-985
• 15 U.S Census Bureau. 2019 ACS 1 year estimates. Explore census data. Accessed March 13, 2022 at: https://data.census.gov/cedsci/table?q=median%20income&g=0400000US36&tid=ACSST1Y2019.S1901
• 16 U.S Census Bureau. 2020 ACS 5 year estimates. Explore census data. Accessed March 13, 2022 at: https://data.census.gov/cedsci/table?q=poverty%20rate%20&g=0500000US36005,36047,36061,36081
• 17 Myers C, Olson C, Kerker B, Thorpe L, Greene C, Farley T. Reducing Health Disparities in New York City: Health Disparities in Life Expectancy and Death. New York, NY: New York City Department of Health and Mental Hygiene; 2010
• 18 Black JL, Macinko J, Dixon LB, Fryer Jr GE. Neighborhoods and obesity in New York City. Health Place 2010; 16 (03) 489-499
• 19 Gresia V, Lundy De La Cruz N, Jessup J. et al. Hypertension in New York City: Disparities in Prevalence. Epi Data Brief No. 82. New York, NY: New York City Department of Health and Mental Hygiene. December 2016. Accessed April 1, 2022 at: https://www1.nyc.gov/assets/doh/downloads/pdf/epi/databrief82.pdf
• 20 Kang JX, Levanon Seligson A, Dragan KL. Identifying New York City neighborhoods at risk of being overlooked for interventions. Prev Chronic Dis 2020; 17: E32
• 21 Karpati A, Kerker B, Mostashari F. et al. Health Disparities in New York City. New York, NY: New York City Department of Health and Mental Hygiene; 2004
• 22 Feinstein MM, Niforatos JD, Mosteller L, Chelnick D, Raza S, Otteson T. Association of Doximity ranking and residency program characteristics across 16 specialty training programs. J Grad Med Educ 2019; 11 (05) 580-584
• 23 Veterans Affairs. Affiliate resources. Published January 6, 2021. Accessed April 19, 2022 at: https://www.va.gov/oaa/affiliation-agreements.asp
• 24 USnews.com. 2023 best medical schools. Accessed April 15, 2022 at: https://www.usnews.com/best-graduate-schools/top-medical-schools

Publication History

Received: 24 October 2022

Accepted: 04 April 2023

Article published online:
17 May 2023