The Use and Evaluation of an Inexpensive Eye Model in Direct Ophthalmoscopy Training

Abstract

Purpose Learning direct ophthalmoscopy to view the fundus is an important skill that is often given insufficient attention in medical school curriculums. The goals of this study were to (1) create an easy-to-assemble, inexpensive artificial eye model, the FAK-I, (2) develop an objective evaluation tool to assess the students' clinical skills in using the direct ophthalmoscopy, and (3) determine if practice with the FAK-I translates into improved clinical examination skills.

Methods After an initial hands-on tutorial on direct ophthalmoscopy, study subjects (n = 12) were randomly assigned to a control group (n = 6), which received no FAK-I model, or a study group (n = 6), which received a FAK-I model. After 45 days of no further intervention, students took an objective evaluation of clinical direct ophthalmoscopy skills. Scores of students in the control and study group were compared.

Results The study group had a higher percentage of correct responses (44%) compared with the control group (40%), but the difference did not reach statistical significance. More students in the study cohort (50%) reported independently practicing the direct ophthalmoscope than the control group (0%). The study cohort also reported practicing on the FAK-I more frequently than on peers or patients.

Conclusions We were able to create the FAK-I, a simple low-cost eye model to practice direct ophthalmoscopy. We also designed an objective evaluation tool to assess students' clinical skills with a direct ophthalmoscope. While the study group scored slightly better than the control group, these numbers did not reach significance due to the limited size of this pilot study. The study group practiced more than the control group, and more often on the FAK-I model than on peers or patients. These findings in this pilot study point to the potential utility of providing a simple eye model in improving direct ophthalmoscopy skills in medical students.

Keywords

educational tools - medical student education - direct ophthalmoscopy - eye model - clinical skills

Proper use of the direct ophthalmoscope to view the fundus is an important skill not only for ophthalmologists, but also for all primary care physicians.[1] [2] However, current medical school curriculums may not permit satisfactory training of medical students in this critically important skill.[1] [3] [4] Literature suggests that medical students are failing to perform or even attempting to perform adequate eye examinations, including proper direct ophthalmoscopy.[1] [4] The need for additional attention to ophthalmology in the medical curriculum has been made clear.[1] [4] [5] Other studies have shown that increased practice with direct ophthalmoscopy can increase self-confidence and ability in its use.[4] [6]

Eye models have been previously developed as an educational tool for direct ophthalmoscopy.[4] [7] [8] [9] [10] The models described by Dodaro and Lippa are similar in that they use a Styrofoam head (The Dow Chemical Company, Midland, Michigan), a slide holder to house retinal images, a model eye, and a plastic face cover.[4] [7] McCarthy used the “EYE Exam Simulator” by Kyoto Kagaku Co., Ltd.,[11] which retails at $1,300 but reported low user satisfaction.[10] Bradley and Hoeg designed eye models using painted table tennis balls and canisters with photos, neither of which was mounted into a model head.[9] The model we describe is significantly less expensive than the Kyoto Kagaku model, more easily assembled than those of Dorado, Lippa, and McCarthy, and has the added advantage of functioning as a stand-alone unit or being mounted in a Styrofoam head to simulate a patient encounter. While several studies evaluated the effectiveness of using eye models as an educational tool, none have done so using a prospective cohort with a control group.[4] [8] [9]

The primary objectives of this study were to (1) create a new artificial eye model that is simple and inexpensive, (2) develop an objective evaluation tool assessing medical students' clinical skills in direct ophthalmoscopy, and (3) use this objective evaluation to determine if practice with the eye model translates into improved clinical examination skills.

A secondary objective was to determine whether access to the model impacts the frequency with which students practice direct ophthalmoscopy on either the model or human subjects.

Methods

Eye Model

The FAK-I model was designed to approximate certain anatomic properties of the human eye, using low-cost and easily assembled materials. An opaque, plastic bottle cap measuring 24 mm in depth was used to replicate the anterior-posterior dimension of the human eye. High-resolution, digital retinal images were created from montage color fundus photography taken by the Topcon 50ix camera and processed by Ophthavision 3.0 (Escalon Medical Corp., Ardmore, Pennsylvania), with the optic nerve scaled to a vertical diameter of 1.5 mm ([Fig. 1]). These photographs, printed on matte paper to minimize glare, were cut to size and placed into the base of the bottle cap ([Fig. 2], inset). Black card stock with a central opening was used to cover the cap, creating an “iris” and “pupil.” Superior poles of eye models were marked with an L or R for placement orientation. The FAK-I model may be used as a stand-alone unit or mounted into a Styrofoam head (available at beauty supply stores) with orbit cutouts, simulating the facial anatomy and practical challenges of patient examination ([Fig. 2]).

Fig. 1 Sample image of fundus photograph.

Fig. 2 Assembled FAK-I with fundus photo inside bottle cap and black card stock cover (inset). FAK-I mounted in Styrofoam head.

Student Participants

This study was approved by the Northwestern University IRB office. Students were recruited from a group of 40 third-year Northwestern University Feinberg School of Medicine students rotating through their surgical clerkship at the time of the study. As first-year medical students, all had previously participated in a 30-minute lecture on eye examination techniques followed by small-group, 90-minute practical training that included direct ophthalmoscopic examination of their peers and a standardized patient. No objective measurement of students' ability to use the direct ophthalmoscope was made during this session, and no other educational interventions for research purposes had been performed prior to their surgical clerkship.

Eight of the 40 students had been previously assigned an elective 2-week ophthalmology rotation during their surgical clerkship, and were excluded from the study. For the remaining 32 students, enrollment was voluntary and did not affect their surgical clerkship evaluations.

Prior to commencement of the study, all 40 students were briefed on its nature and purpose, and all participated in a 15-minute hands-on direct ophthalmoscopy training session with three instructors. Of the 32 potential enrollees, 19 students volunteered to participate in the study, 12 of whom were randomly selected. These 12 students were further divided by randomly assigning de-identified participant forms into the control arm (no FAK-I model intervention) and study arm (FAK-I model intervention) with 6 students in each group. Students enrolled into the study arm received a FAK-I set consisting of a Styrofoam head mount and five FAK-I eye models depicting different posterior pole findings. The investigators did not provide students with direct ophthalmoscopes, but they were told where direct ophthalmoscopes were available throughout the medical school campus.

The student intervention period was 45 days in length, and concluded with the objective evaluation. During this period, all 40 students attended a required ophthalmic practical skills session, consisting of two hours of lectures followed by small-group sessions designed to practice complete eye examinations. Study participants received no additional training, and investigator contact with student enrollees was limited to email communication regarding logistics for the final evaluation.

Patient Participants

Patients were recruited on a voluntary basis from the Northwestern Medical Faculty Foundation Ophthalmology Clinic. Inclusion criteria were patients over the age of 18 with a clear view of the posterior pole. Pregnant patients and patients unable to tolerate 12 consecutive fundus exams for any physical, emotional, or psychological reason were excluded.

Evaluation Tool

To objectively evaluate students' ability to examine a patient with the direct ophthalmoscope in a clinical setting, an evaluation tool similar to that used in United States Medical Licensing Examination Step 2 Clinical Skills was designed. Students enrolled in this study had previous exposure to similar methods during their medical school training.

After 12 patient subjects were recruited, one of two investigators (RM or AB) performed a bilateral dilated exam, selecting one eye per patient to be photographed for later use in the objective evaluation. On the day of the evaluation, patients had only the predesignated eye dilated using tropicamide 1% and phenylephrine 2.5%. Each patient's fundus photograph was printed among five other unrelated photographs selected from the Northwestern departmental photographic database. Students evaluated the dilated eye of each patient in successive 90-second encounters. For each patient encounter, students were instructed to introduce themselves, wash their hands, examine the posterior pole using direct ophthalmoscopy, select the matching fundus photo from six possible photographic choices, and record their selection on their answer sheet. No patient information was provided to student examiners and no other examination techniques were permitted or performed. Additionally, students were not asked to diagnose ocular pathology and no photographs were labeled.

Following the session, students completed an anonymous questionnaire inquiring about prior experience and use of the direct ophthalmoscope, estimated time spent using the FAK-I models over the course of the study, estimated time spent practicing direct ophthalmoscopy on a patient or peer over the course of the study, ownership of a direct ophthalmoscope, intended field of postgraduate training, and expected use of the direct ophthalmoscope as a practicing physician.

Students were scored one point for the correct identification of a patient's fundus photograph, and percent correct answers were then calculated.

Statistical Analysis

The study was powered to detect a 23% difference in the ability of the study cohort to correctly identify patients' fundus photos from their exam when compared with the control cohort. A total of 144 assessments were made by 12 students (6 study cohort students and 6 control cohort students) examining 12 patients. The decision to power the study as such was in part affected by the number of dilated examinations the investigators felt a typical patient could tolerate within a relatively short period of time. Fisher exact test was used to compare the demographic characteristics of the study and control groups as well as the secondary study outcome. Assessment of the primary outcome was performed using a generalized linear model.

Results

Student Demographics

Of the students in the study, all 12 (100%) reported prior use of direct ophthalmoscopy during previous clinical rotations. In both cohorts, five (83%) of six students anticipated using direct ophthalmoscopy in the future. There was great variability among both cohorts regarding their future medical specialty choice, with no students citing a traditional surgical field. More students owned a direct ophthalmoscope in the study group (two, 33%) compared with the control group (zero, 0%), but more students had prior exposure to ophthalmology in the control group (two, 33%) compared with the study group (zero, 0%). The differences between the two cohorts with regard to both ownership and prior exposure did not reach statistical significance (p > 0.05) ([Table 1]).

Table 1 Demographic data

	Control (n [%])	Study (n [%])	p value
Prior training	3 (50)	4 (67)	NS[a]
DO ownership	0 (0)	2 (33)	NS
DO use in prior rotation	6 (100)	6 (100)	NS
Future medical specialty
• Internal medicine and subspecialty	2 (33)	3 (50)
• Pediatrics	0 (0)	1 (17)
• Family medicine	1 (17)	1 (17)
• Emergency medicine	1 (17)	0 (0)
• Anesthesiology	1 (17)	1 (17)
• Unknown	1 (17)	0 (0)
Anticipated DO use	5 (83)	5 (83)	NS
Prior exposure to ophthalmology	2 (33)	0 (0)	NS

Abbreviation: DO, direct ophthalmoscopy.

^a NS indicates p > 0.05 between control and study groups.

Primary Outcomes

All students from the study (FAK-I) cohort completed all 12 patient encounters as planned, while only 5 of the control cohort were able to finish all encounters. One student from the control group completed 11 patient encounters, but was unable to complete the 12th due to malfunction of the direct ophthalmoscope. An intention-to-treat approach was taken in the statistical analysis of the number of correct responses, scoring the incomplete encounter as a correct response ([Table 2]).

Table 2 Primary outcome

	Control (n [%])	Study (n [%])	p value
Correct responses,[a] intention to treat	29 (40)	32 (44)	0.61

^a Numbers shown reflect total number of correct responses in the group out of 72 possible responses.

The average number of correct answers from the control cohort was 5.0 compared with 5.3 (range 1–8) in the study cohort. There was a slight trend toward higher percentage of correct responses in the study group compared with the control group (44 vs 40%, respectively); however, the difference between the groups did not reach statistical significance (p = 0.61).

Secondary Outcomes

More subjects in the study cohort reported independent practice using the direct ophthalmoscope (three, 50%) compared with the control cohort (zero, 0%); however, these numbers did not reach statistical significance (p = 0.1). Students in the study cohort reported spending more time practicing on the FAK-I model (three, 50%) than on peers or patients (one, 16.7%). No student reported spending more than 90 minutes (range 15–90 minutes) practicing on the FAK-I or on peers or patients over the course of the study ([Table 3]).

Table 3 Secondary outcome

	Control (n [%])	Study (n [%])	p value
Any independent practice during surgical rotation	0 (0)	3 (50)	>0.1
Practice on peers or patients	0 (0)	1 (17)
Practice on FAK-I	N/A	3 (50)

Discussion

We have employed two innovative methods to achieve our main objectives. First, we created a new artificial eye model, FAK-I, which is simple and inexpensive to construct. Using a bottle cap, printed fundus images, card stock, and a Styrofoam head, a training set may be assembled for less than $20 (plus photographic costs). It also provides anatomically realistic attributes for simulated fundus examination in the settings of both individual practice and group instruction. Second, we employed a novel objective evaluation tool to assess medical students' clinical skills in direct ophthalmoscopy by using the same 12 patients for all subject testing within a controlled setting.

This study looked at the effects of providing a simple-to-use tool, the FAK-I, to medical students in a clinical rotation, independent of any other intervention. Important conclusions can be drawn from this study despite the fact that the results did not reach statistical significance. First, the study suggests that the majority of medical students anticipate using direct ophthalmoscopy in their future practice. Second, simply providing the resources for independent practice of direct ophthalmoscopy on an eye model does not necessarily translate into actual use of the model. Only 50% of the students in the study cohort practiced with FAK-I without supervision. It may be that perceived limitations in time and the inconvenience of obtaining a direct ophthalmoscope with which to practice were barriers to model use. Although a majority of students recognized the need to perform direct ophthalmoscopy in the future, varying levels of motivation may be another contributing factor to the lack of practice. Third, more students from the study group reported practicing on the FAK-I as opposed to peers or patients, suggesting that students may find it more convenient and comfortable to practice on a model. Finally, results also show a positive trend in the frequency of independent practice with the direct ophthalmoscope among the students of the study cohort in comparison to the control cohort. In addition, the study group also practiced more on peers and patients than the control group, suggesting that availability of the FAK-I model may have stimulated their interest to practice on human subjects. However, this study was underpowered to detect a statistical significance for differences on these points.

In this small pilot study, simply providing the FAK-I model to students without further intervention did not significantly improve competency with direct ophthalmoscopy in the objective evaluation. An investigation with a larger and more comprehensive study population should be conducted before any further conclusions are made. While this study was limited to students in their third-year surgical rotation, the FAK-I model could certainly be an effective tool for medical students to practice their eye examination skills at any point in their training.

This study had several limitations. The study size was limited due to what was considered to be a tolerable number of dilated exams for a patient to undergo in a short period of time. As a result, the study was underpowered to detect a small increase, if present, in the ability of the study cohort to correctly identify patient fundus photography when compared with the control cohort.

The performance of the students on the objective evaluation also may have been affected by several confounding variables. Difficulty level of each set of photographs based on similarity or dissimilarity between the correct and incorrect photographs is difficult to standardize. Also, differences in the patient interactions, such as patient compliance or variability of the equipment in the examination rooms, may have positively or negatively impacted the students' performance during the objective examination.

While the findings of this study did not reach statistical significance, the trends toward improved performance on the objective examination and increased likelihood of independent practice in the study population support the utility of the FAK-I model as a tool for practicing direct ophthalmoscopy.

Grants

Unrestricted grant from Research to Prevent Blindness, New York.

Acknowledgments

We thank Evica Simjanoski, CRA; Alfred Rademaker, PhD; Juned Siddique, PhD; Jie Peng, MS; Rajen Desai, MD; Safa Rahmani, MD; Nicholas Volpe, MD.

• References

• 1 Stern GA ; The Association of University Professors of Ophthalmology Education Committee. Teaching ophthalmology to primary care physicians. Arch Ophthalmol 1995; 113 (6) 722-724
• 2 Worthen DM. Ophthalmology for medical students: objectives. Arch Ophthalmol 1972; 88 (3) 314-315
• 3 McNaught AI, Pearson RV. Ownership of direct ophthalmoscopes by medical students. Med Educ 1992; 26 (1) 48-50
• 4 Lippa LM, Boker J, Duke A, Amin A. A novel 3-year longitudinal pilot study of medical students' acquisition and retention of screening eye examination skills. Ophthalmology 2006; 113 (1) 133-139
• 5 Quillen DA, Harper RA, Haik BG. Medical student education in ophthalmology: crisis and opportunity. Ophthalmology 2005; 112 (11) 1867-1868
• 6 Gupta RR, Lam WC. Medical students' self-confidence in performing direct ophthalmoscopy in clinical training. Can J Ophthalmol 2006; 41 (2) 169-174
• 7 Dodaro NR, Maxwell Jr DP. An eye for an eye. A simplified model for teaching. Arch Ophthalmol 1995; 113 (6) 824-826
• 8 Bradley P. A simple eye model to objectively assess ophthalmoscopic skills of medical students. Med Educ 1999; 33 (8) 592-595
• 9 Hoeg TB, Sheth BP, Bragg DS, Kivlin JD. Evaluation of a tool to teach medical students direct ophthalmoscopy. WMJ 2009; 108 (1) 24-26
• 10 McCarthy DM, Leonard HR, Vozenilek JA. A new tool for testing and training ophthalmoscopic skills. J Grad Med Educ 2012; 4 (1) 92-96
• 11 Eye Examination Simulator. Available from http://limbsandthings.com/us/products/eye-examination-simulator/ . Accessed January 19, 2013