Comparing the Diagnostic Utility of Conventional Direct Ophthalmoscopy with Smartphone Ophthalmoscopy among Medical Students

Abstract

Purpose To compare the diagnostic ability of medical students using smartphone ophthalmoscopy (SO) with conventional direct ophthalmoscopy (DO).

Methods Twenty-eight first- and second-year medical students were trained to use the SO and DO. They also attended educational seminars regarding optic nerve and retinal pathology and were given hands-on practice with each ophthalmoscopy method. Students were randomized 2:1 into one of the groups (DO or SO). Students then examined six patients and recorded their findings, ease of use, and confidence level on a questionnaire. Two attending ophthalmologists, masked to the randomization, graded the student questionnaires. A priori power calculation determined the sample size. The primary outcome measure was the percentage of correct diagnoses the students made. Two-sample t-test, Wilcoxon's rank-sum test, and Fisher's exact test were used to compare the outcomes.

Results Students using the SO outperformed students using DO in terms of mean percent correct (% correct) diagnosis (smartphone: 42% vs. direct: 23%; p-value = 0.0057), mean % correct photo match (smartphone: mean = 60% vs. direct: 32%; p-value = 0.0052), and mean % correct nerve/retinal descriptors (smartphone: 72% vs. direct: 59%; p-value = 0.0048). There was not a significant difference in terms of perceived ease of use (smartphone: mean = 3.3 vs. direct: mean = 2.6; p-value = 0.0945), or subjective confidence (smartphone: mean = 2.6 vs. direct: mean = 2.1; p-value = 0.0808) between the two groups.

Conclusion SO provides an alternate way for medical students to learn, diagnose, and describe ocular pathology.

Keywords

smartphone ophthalmoscopy - direct ophthalmoscopy - medical student - ophthalmic education - funduscopy

Ophthalmic medical student education poses unique challenges as it requires students to learn and diagnose ocular pathology using complex technical skills such as direct ophthalmoscopy (DO).[1] While many ophthalmology departments teach students to use DOs for funduscopy, it is often challenging for students to master this skill in the short time allotted to ophthalmic clinical education in medical school.[2] Students often lack confidence in their ability to view the fundus and cannot correctly diagnose ocular pathology.[2] [3] [4] [5] Unfortunately, this leads to physicians being uncomfortable with DO and may subsequently compromise the ability to triage patients with ocular or neurologic complaints.[6]

In recent years, many digital ophthalmoscopes (DGO) have become available that provide an easier method of viewing the fundus for health care professionals, including the D-EYE (2016 D-EYE Srl, Padova, Italy). However, the downsides of DGOs are cost, portability, and accessibility.[7] Smartphone ophthalmoscopes (SOs) were created as a cost-effective alternative by transforming a smartphone camera into an ophthalmoscope using modular attachments and software.[8] While the SO can capture good photographic and video quality, adopting an SO in ophthalmic practice requires further improvement in image resolution. However, current literature does support that the SO can be an effective tool for fundus evaluation and diagnosis of retinal pathology.[8]

Despite its limitations, the SO can provide an interesting alternative to medical student education. Medical students are an ideal demographic for combining this clinical skill with modern technology smartphones.[1] [9] The ability for students to record and revisit their fundus examinations is a promising tool to help them improve their ophthalmoscopy skills. Additionally, the functionality of Sos allows instructors to provide real-time instruction to help students learn and improve their skills. This study aims to determine whether the SO is an appropriate alternative to the DO by comparing the diagnostic ability of students using the two devices.

Methods

This study was a randomized, prospective trial conducted at Dean McGee Eye Institute (DMEI), University of Oklahoma Medical Center (OUMC). It was approved by the OUMC Institutional Review Board. All medical information retrieved for this study were reviewed in compliance with the Health Insurance Portability and Accountability Act. All facets of this study were conducted per the tenets of the Declaration of Helsinki.

All patients (n = 6) included in this study were treated at DMEI and consented to be a part of this study. Students examined a single assigned eye for each patient (five ODs and one OS). These patients/eyes were chosen for their highly characteristic findings for the following diagnoses: (1) proliferative diabetic retinopathy, (2) primary open-angle glaucoma, (3) myopic tilt, (4) optic nerve atrophy, (5) hypertensive retinopathy, and (6) papilledema. Before the student examinations, the patient's chosen eye was dilated with phenylephrine 2.5% and tropicamide 1%. During the study, patients were checked periodically to confirm that their eyes remained adequately dilated for the student examinations.

All first- and second-year medical students (n = 273) at the University of Oklahoma College of Medicine were invited to participate in the study. A total of 28 students (18 males and 10 females) responded and agreed to participate. The group consisted of 8 first-year and 20 second-year medical students ([Table 1]). The study participants signed a copy of informed consent. Before the examination, students attended a training session consisting of a 30-minute lecture regarding the retinal/optic nerve pathology found in the study patients and the use of both methods of ophthalmoscopy. Following the lecture, attendings supervised the students as they practiced their ophthalmoscopy skills with both methods until they felt sufficiently comfortable using them. Students were assigned a study number to maintain their anonymity and were assigned to a group via 1 SO:2 DO randomization. Students assigned to the smartphone group received a D-EYE system paired with an Apple iPhone 7 (Foxconn/Pegatron, China). Following their dilated patient examination, students completed a questionnaire reporting their diagnosis, findings, confidence level, and perceived ease of use with their assigned ophthalmoscope method ([Fig. 1]). The questionnaire also included fundus photographs of the patient's eyes taken 1 to 2 days before the study, and students were instructed to match each image with the correctly corresponding patient examination ([Fig. 2]). The completed questionnaires were then graded based on an answer key developed by two ophthalmologists independently. The ophthalmologists agreed 100% on their answer key responses. Before the study, power calculations were performed. Based on prior experience, a medical student educator estimated the percentage of correct answers for students in the SO and DO groups. Based on these assumptions, the estimated standard deviation in the correctness score (out of a total score of 6) was 0.94 for the smartphone group and 1.04 for the conventional group. Using a two-sample t-test (0.05 α level, two sided) with unequal variance, 9 participants in the smartphone group and 18 participants in the conventional group achieved 94.5% power to detect a difference of 1.5 in the correctness score between the smartphone and conventional groups.

Fig. 1 A printed copy of the online questionnaire medical students completed.

Fig. 2 Comparison of Optos photography to student smartphone examination—(a) Patient 2 (primary open angle glaucoma), (b) student examination of Patient 2, (c) Patient 3 (normal optic disc), and (d) student examination of Patient 3.

Table 1 Characteristics of students involved in the study

Characteristics	Direct (n = 18)	Smartphone (n = 10)
Age (y)	31.3	26.0
Year of medical school
MS1	5	3
MS2	13	7
Sex
Male	12	6
Female	6	4

Abbreviations: MS1, first-year medical student; MS2, second-year medical student.

The primary outcome measure was the student's ability to diagnose the study patients correctly. The percent correct (% correct) was also determined for descriptors and photo matching. Secondary outcomes included students' perceived ease of use and confidence level with each method. SAS 9.4 was used to perform the analysis. An α level of 0.05 (two sided) was used for hypothesis testing. % correct, percent unable to observe (% UtO), and satisfaction scores (ease of use, confidence) were analyzed between the two groups for several factors. Two-sample t-test, Wilcoxon's rank-sum test, and Fisher's exact test were used to compare continuous, ordinal, and categorical outcomes, respectively. Spearman's correlation coefficient was used to assess the correlation between satisfaction scores (ease of use, confidence) and % correct in diagnosis and photo match.

Results

The mean % correct by students using the SO was 42% compared with students using the DO, who scored a mean of 23% (p-value = 0.0057). In terms of correctly matching their clinical examination findings to representative images of the patients' fundus photos, students using the SO outperformed students using the DO with a mean of 60% correct compared with a mean of 32% (p-value = 0.0052) ([Table 2]). Furthermore, students using the SO had a higher mean score when describing the optic nerve and retinal vasculature with 72%, compared with DO users with a mean of 59% (p-value = 0.0048) ([Table 2]). When separated into individual scores for the optic nerve and retinal vasculature descriptors, there was still a significant difference between the two methods ([Table 2]).

Table 2 Examination results for direct ophthalmoscopy versus smartphone ophthalmoscopy

	Direct, mean (std)	Smartphone, mean (std)	p-Value	Test
Percent correct diagnosis	0.23 (0.14)	0.42 (0.18)	0.0057	t-test
Percent correct photo match	0.32 (0.21)	0.60 (0.26)	0.0052	t-test
Percent correct combined descriptors	0.59 (0.16)	0.72 (0.07)	0.0048	t-test
Percent correct nerve descriptors	0.59 (0.19)	0.73 (0.09)	0.0083	t-test
Percent correct vasculature descriptors	0.59 (0.16)	0.71 (0.07)	0.0075	t-test
Ease of use	2.61 (0.78)	3.30 (1.25)	0.0945	Wilcoxon's rank-sum
Confidence	2.06 (0.80)	2.60 (0.84)	0.0808	Wilcoxon rank-sum
% UtO nerve descriptors	0.21 (0.23)	0.05 (0.08)	0.0305	Wilcoxon's rank-sum
% UtO vasculature descriptors	0.18 (0.21)	0.05 (0.08)	0.0216	Wilcoxon's rank-sum

Abbreviations: % UtO, percent unable to observe; std, standard deviation.

Students also recorded when they were unable to observe the optic nerve or the vasculature. There was a significant difference in % UtO between the groups for both nerve (p-value = 0.0305) and vasculature (p-value = 0.0216) ([Table 2]); however, when analyzing the difference for each patient, it was only significant for Patient 1, who has proliferative diabetic retinopathy ([Table 3]). For this patient, 61% (n = 11) of the DO group reported being unable to view the optic nerve versus 0% (n = 1) in the SO group and 56% (n = 10) in the DO group were unable to view the retinal vasculature versus 10% (n = 1) in the SO group ([Table 3]). For all other patients, there was no significant difference between the two groups regarding the ability to view the optic nerve or the retinal vasculature ([Table 3]).

Table 3 Results of students unable to observe the nerve or vasculature per patient

	Direct, % (count)	Smartphone, % (count)	p-Value	Test
Patient 1, UtO for nerve	0.61 (11)	0 (0)	0.0016	Fisher's exact
Patient 2, UtO for nerve	0.11 (2)	0 (0)	0.5238	Fisher's exact
Patient 3, UtO for nerve	0.11 (2)	0 (0)	0.5238	Fisher's exact
Patient 4, UtO for nerve	0.22 (4)	0.10 (1)	0.6264	Fisher's exact
Patient 5, UtO for nerve	0.11 (2)	0.20 (2)	0.6015	Fisher's exact
Patient 6, UtO for nerve	0.11 (2)	0 (0)	0.5238	Fisher's exact
Patient 1, UtO for vasculature	0.56 (10)	0.10 (1)	0.0407	Fisher's exact
Patient 2, UtO for vasculature	0.11 (2)	0 (0)	0.5238	Fisher's exact
Patient 3, UtO for vasculature	0.60 (1)	0 (0)	1.0000	Fisher's exact
Patient 4, UtO for vasculature	0.22 (4)	0.10 (1)	0.6264	Fisher's exact
Patient 5, UtO for vasculature	0.60 (1)	0.10 (1)	1.0000	Fisher's exact
Patient 6, UtO for vasculature	0.17 (3)	0 (0)	0.5330	Fisher's exact

Abbreviation: UtO, unable to observe.

There was no significant difference found between the groups for perceived ease of use (p-value = 0.0945) or level of confidence (p-value = 0.0808) ([Table 2]). However, when analyzing participants from both groups together, significant correlations were found when comparing ease of use to both % correct diagnosis (correlation coefficient = 0.46, p-value = 0.0140) and % correct photo match (correlation coefficient = 0.39, p-value = 0.0375) ([Table 4]). There was no significant correlation between confidence levels versus % correct diagnosis and photo match ([Table 4]).

Table 4 Correlation analysis between perceived ease of use and confidence to percentage of correct diagnosis and photo match

	Percent correct diagnosis, correlation coefficient (p-value)	Percent correct photo match, correlation coefficient (p-value)
Ease of use	0.46 (0.0140)	0.39 (0.0375)
Confidence	0.21 (0.2823)	0.21 (0.2875)

Discussion

DO is a critical skill that assists physicians in primary care or emergency department settings in diagnosing, triaging, and treating patients with ocular disease.[1] [3] [4] [7] While DO is taught at ∼84% of medical schools across the country, the limited amount of time allotted is insufficient to gain competency in this difficult and technical skill.[1] [2] Many students and physicians alike lack confidence in their abilities to use DO to diagnose and treat patients.[2] [5] [6] [10] Unfortunately, learning DO requires considerable time and practice.[7] Another challenge is feedback students receive during instruction which relies on what students can see or not see.[7]

The goal of this study was to compare SO with DO to determine if this alternative method helps students learn, diagnose, and describe ocular pathology by comparing their diagnostic skills. Our data suggest that students have significantly better diagnostic skills using the SO than the DO. The SO group achieved a higher correct percentage in diagnosis, photo match, and description. This study did not show a significant difference in perceived ease of use and confidence. This finding was an interesting result because similar studies have reported that students had a more positive experience with the SO, specifically regarding ease of use and confidence levels.[1] [9] [11] One possibility that students in our study did not show this finding may be because they only used one ophthalmoscopy method for their examinations and thus could not directly compare their experiences with the other method. Another possible explanation is that the study's sample size may not be large enough to detect differences in these variables.

In our study, students preferred the SO over the DO for many reasons, such as the ability to review their clinical examinations at their leisure and have real-time feedback during practice sessions. Anecdotally, some students also mentioned that it was the first time that they were able to confirm that they were visualizing the optic disc and retinal vasculature. Many students also said that patient factors, such as more light sensitivity with DO versus SO, can affect their examination. We hypothesize that SO may also benefit students in examining light-sensitive patients due to dimmer light from the smartphone flash, which may have led to the significant difference in our data.

A limitation of this study was that the medical students' voluntary participation might overly motivate our students to learn ophthalmoscopy more than other students. Additionally, students were taught to recognize limited pathology, and the selected patients were highly representative of the condition. Finally, procurement of the D-EYE system was difficult; this hindered our ability to increase the number of units used in the study and defend against technical difficulties that could influence the results.

Conclusion

In summary, SO is a valuable tool that improves medical students' ability to view, describe, and diagnose ocular pathology correctly. Additionally, these new technologies can provide avenues to enhance ophthalmic education for medical students. A longer term study that assesses retention of the diagnostic and practical skills of the students learned would be beneficial as an alternative teaching method.

Conflict of Interest

None declared.

• References

• 1 Kim Y, Chao DL. Comparison of smartphone ophthalmoscopy vs conventional direct ophthalmoscopy as a teaching tool for medical students: the COSMOS study. Clin Ophthalmol 2019; 13: 391-401
• 2 Shah M, Knoch D, Waxman E. The state of ophthalmology medical student education in the United States and Canada, 2012 through 2013. Ophthalmology 2014; 121 (06) 1160-1163
• 3 Gupta RR, Lam WC. Medical students' self-confidence in performing direct ophthalmoscopy in clinical training. Can J Ophthalmol 2006; 41 (02) 169-174
• 4 Roberts E, Morgan R, King D, Clerkin L. Funduscopy: a forgotten art?. Postgrad Med J 1999; 75 (883) 282-284
• 5 Mackay DD, Garza PS, Bruce BB, Newman NJ, Biousse V. The demise of direct ophthalmoscopy: a modern clinical challenge. Neurol Clin Pract 2015; 5 (02) 150-157
• 6 Dalay S, Umar F, Saeed S. Fundoscopy: a reflection upon medical training?. Clin Teach 2013; 10 (02) 103-106
• 7 Taylor C. Digital ophthalmoscopy: through a non-specialist lens. Future Healthc J 2021; 8 (01) e76-e78
• 8 Dickson D, Fouzdar-Jain S, MacDonald C. et al. Comparison study of funduscopic exam of pediatric patients using the D-EYE method and conventional indirect ophthalmoscopic methods. Open J Ophthalmol 2017; 7 (03) 145-152
• 9 Wu AR, Fouzdar-Jain S, Suh DW. Comparison study of funduscopic examination using a smartphone-based digital ophthalmoscope and the direct ophthalmoscope. J Pediatr Ophthalmol Strabismus 2018; 55 (03) 201-206
• 10 Schulz C, Hodgkins P. Factors associated with confidence in fundoscopy. Clin Teach 2014; 11 (06) 431-435
• 11 Nagra M, Huntjens B. Smartphone ophthalmoscopy: patient and student practitioner perceptions. J Med Syst 2019; 44 (01) 10

Publication History

Received: 16 August 2022

Accepted: 17 January 2023

Article published online:
01 March 2023