Teaching Ophthalmology Residents Clinical Optics Via a Flipped Classroom Curriculum

Abstract

Background Clinical optics is an essential part of ophthalmology resident education that can be challenging for both learners and teachers when taught using a lecture format. The effectiveness of a flipped classroom approach in this context has not been formally evaluated.

Objective The main purpose of this article is to compare the effectiveness of flipped classroom versus lecture-based clinical optics curricula in a graduate medical education setting.

Design Retrospective, nonrandomized, pre- and post-interventional study from 2009 to 2016.

Setting, Participants Ophthalmology residency program at the Casey Eye Institute, Oregon Health & Science University, an academic medical center in the United States. Participants included all ophthalmology residents able to take at least one Ophthalmic Knowledge Assessment Program (OKAP) examination during the years 2009 to 2016.

Methods The clinical optics curriculum was changed from a lecture-based series to a flipped classroom curriculum and moved from the fall to winter during the 2012 to 2013 academic year. No major changes were made to the curriculum in other subject areas during the study period. Resident performance on the OKAP annual national in-service examination for the 4 years before and after the optics curriculum change was compared. Specifically, the scaled subtest scores from the Optics, Refraction, and Contact Lens subsection were examined, while scores from the 10 nonoptics subsections served as controls.

Results Scores from 57 resident test administrations before the optics curriculum change and 59 after the optic curriculum change were available for comparison. The Optics, Refraction, and Contact Lens subsection mean scores were 50.37 ± 2.31 and 57.27 ± 2.47 before and after the optics curriculum change, respectively (mean ± 95% confidence interval). This was the only subsection score to show a statistically significant difference after the optics curriculum change (p = 0.00008).

Conclusions and Relevance In comparison to a lecture-based curriculum, a flipped classroom approach to clinical optics education was found to be associated with higher ophthalmology resident performance on the optics subsection of the OKAP examination. Our study suggests that a flipped classroom format may be more effective than traditional lectures for teaching clinical optics in a graduate medical education setting.

Keywords

optics - flipped classroom - problem-based learning - ophthalmology - residency - medical education

Ophthalmologists must demonstrate competence in clinical optics to attain professional certification by the American Board of Ophthalmology. Unfortunately, the traditional lecture-based method of instructing ophthalmology residents in optics has anecdotally in our experience been perceived by learners and instructors to be both challenging and of low educational yield. Alternative active learning models such as the flipped classroom approach may help to address these concerns.[1]

The flipped classroom method involves learners performing independent study outside of the classroom before attending in-class sessions. During in-class sessions, learners participate actively by teaching the material covered during independent study to their peers.[1] Key elements of the flipped classroom approach were first introduced as a “peer instruction” method in 1991 by Eric Mazur at Harvard University.[2] Over the next few decades, variations in this curricular approach were popularized and applied to medical education as the “flipped classroom” method, although there did not exist clear evidence that this pedagogical approach would improve the competency of healthcare providers. More recently, there has been building evidence that flipped classroom curricula can be more effective than lecture format in some medical education settings.[1] [3] However, meta-analyses thus far comparing the efficacy of flipped classroom and lecture-based medical curricula have not conclusively weighed in favor of the flipped classroom methodology.[1] [3] [4] [5] It has been difficult to ascertain the impact of flipped classroom curricula on learners in part due to variability in both flipped classroom and research methodology among past studies.[4]

With regard to clinical optics, there have been advocates for an active learning, problem-focused approach to this subject matter as early as 1913.[6] However, the efficacy of an active educational format applied to clinical optics has not been directly examined. We aimed to test whether a flipped classroom approach to clinical optics education can be more effective than traditional lecture-based didactics in facilitating learner acquisition of clinical optics knowledge and improving their ability to apply this information. To our knowledge, this is the first investigation into the implementation of a flipped classroom format to clinical optics in a graduate medical education setting.

Methods

Ophthalmology resident clinical optics instruction at Oregon Health & Science University was delivered via traditional lecture-based didactics until a flipped classroom curriculum was implemented during the 2012 to 2013 academic year and continued thereafter. [Table 1] summarizes the major elements of the two curricula.

Table 1 Comparison of educational curricula

Curriculum type	Lecture-based curriculum	Flipped classroom curriculum
Focus	Instructor-centered	Learner-centered
Learner activities prior to in-class sessions	Learners provided with reference books and recommended reading	Learners provided with reference books, recommended reading, and online video lecture series. Learners complete assigned problem sets independently
Learner and instructor activities during in-class sessions	Lectures delivered by faculty instructors via projected slide presentations. Learners listen to lectures and are allowed to ask questions	Learners take turns teaching each other in small groups. Faculty provide oversight and guidance as needed
Faculty instructor time commitment and preparation	Instructors prepare lecture slide presentations	Instructors review predefined problem set and answer key
Academic years curriculum applied	AY 2008–09 to AY 2011–12	AY 2012–13 to AY 2015–16
Timing of in-class sessions	Fall (September–October)	Winter (January–February)
Mean number of in-class sessions per year	4.5	4.25
Mean number hours dedicated to in-class sessions per year	18.0	11.75

Abbreviation: AY, academic year.

Our lecture-based curriculum consisted of projected slide presentations developed and delivered by faculty. Residents were expected but not required to prepare for lecture by reading the appropriate sections from the Basic and Clinical Science Course (BCSC) reference books. Didactics were delivered in the fall from September to October.

Our flipped classroom approach to optics was structured as follows: 4 sequential problem sets were constructed, in total consisting of 100 optics questions, 42 of which are from ophthalmic optics and clinical refraction by Guyton et al.[7] Learners were asked to independently complete the appropriate set of optics questions prior to each of four mandatory in-class group sessions. Learners were provided with the following reference materials to utilize during their initial independent study period—BCSC Section 3: Clinical Optics, Ophthalmic Optics and Clinical Refraction by David Guyton, Review questions in ophthalmology, and the prerecorded optics video lecture series by David Hunter.[7] [8] [9] [10] We instructed residents to prepare to teach the material covered in the problem sets to their peers. At each in-class session, residents took turns working through the assigned problems and teaching their peers within small groups of four to five learners organized by year of training. Faculty instructors supervised the discussions, answered questions, and corrected mistakes, but did not lead the teaching sessions. In class, flipped classroom sessions were delivered in late winter, from January to February.

We compared resident performance on the Ophthalmic Knowledge Assessment Program (OKAP) annual national in-service examination for the 4 years before and after the optics curriculum change. No other major curricular changes were implemented during this period of time. All OKAP examination scores from 2009 to 2016 were included with no exclusions. Data from each year included scores from 15 residents except 2009 (12 residents) and 2014 (14 residents). The scaled scores from the Optics, Refraction, and Contact Lens subsection were examined with scores from the 10 nonoptics subsections serving as controls. Descriptive statistical analysis was performed using Microsoft Excel (Redmond, WA). Unpaired, two-tail t-tests were performed on the scores from the 11 subsections of the OKAP examination, comparing scores before (2009–2012) and after (2013–2016) the optics curriculum change. Bonferroni correction was applied to adjust the p-value for statistical significance given the 11 subsection comparisons. This analysis was also repeated for each postgraduate year (PGY) class of residents.

Results

The Optics, Refraction, and Contact Lens mean subsection scores were found to be 50.4 ± 2.3 and 57.3 ± 2.5 (mean ± 95% confidence interval [CI]) before and after the optics curriculum change, respectively ([Fig. 1]). Optics, Refraction, and Contact Lens was the only subsection to show a statistically significant difference when comparing scores before and after the optics curriculum change (p = 0.00008, [Table 2]). Descriptive statistical analysis of all 11 OKAP examination subsection scores before and after the optics curriculum change is shown in [Table 2]. Bonferroni correction of the α to account for the multiple comparisons yielded a p value of 0.0046 necessary for statistical significance. Subgroup analysis of each PGY of training showed that PGY-2 and PGY-3 optics scores were higher with flipped classroom (55.9 ± 4.4 and 59.1 ± 4.1, respectively, mean ± 95% CI) compared with lecture-based curricula (46.7 ± 4.4 and 51.1 ± 3.7, respectively, mean ± 95% CI). This difference was statistically significant with p values of 0.0037 for PGY-2 and 0.0043 for PGY-3 residents. There was no statistically significant difference in optics subsection scores for PGY-4 residents when comparing flipped classroom and lecture-based curricula (53.3 ± 5.1 and 56.8 ± 4.1, respectively, mean ± 95% CI; p = 0.25).

Table 2 OKAP subsection performance

OKAP subsection	Before or after optics curriculum change	Mean score	Median score	SD	95% CI	Max	Min	Range	n	p-Value
1. General medicine	Before	49.51	49	9.1	2.4	67	29	38	57	0.0052
	After	54.41	57	9.4	2.5	66	16	50	59
2. Fundamentals and principles of ophthalmology	Before	52.42	53	9.1	2.4	69	32	37	57	0.59
	After	53.29	53	8.3	2.2	68	30	38	59
3. Optics, refraction, and contact lens	Before	50.37	51	8.7	2.3	67	32	35	57	0.000083
	After	57.27	60	9.5	2.5	74	34	40	59
4. Ophthalmic pathology and intraocular tumors	Before	53.72	54	7.0	1.8	68	39	29	57	0.10
	After	55.92	57	7.4	1.9	67	23	44	59
5. Neuro-ophthalmology	Before	53.30	53	8.5	2.2	72	28	44	57	0.074
	After	56.19	57	8.8	2.3	71	27	44	59
6. Pediatric ophthalmology and strabismus	Before	52.33	54	8.5	2.3	68	28	40	57	0.18
	After	54.51	56	8.8	2.3	67	24	43	59
7. Orbit, eyelids, and lacrimal system	Before	54.16	54	6.6	1.8	67	35	32	57	0.88
	After	53.93	56	9.5	2.5	67	7	60	59
8. Cornea, lens, and external disease	Before	51.74	51	8.3	2.2	67	25	42	57	0.25
	After	53.47	54	8.0	2.1	70	30	40	59
9. Intraocular inflammation and uveitis	Before	52.02	52	7.0	1.9	64	23	41	57	0.034
	After	55.22	56	8.9	2.3	70	25	45	59
10. Glaucoma	Before	52.35	52	9.2	2.4	68	31	37	57	0.040
	After	55.97	59	9.6	2.5	70	20	50	59
11. Retina and vitreous	Before	53.93	56	7.7	2.0	68	33	35	57	0.71
	After	53.34	55	9.3	2.4	67	18	49	59

Abbreviation: CI, confidence interval; OKAP, Ophthalmic Knowledge Assessment Program; SD, standard deviation.

Fig. 1 Ophthalmic Knowledge Assessment Program (OKAP) examination scores before and after implementation of a flipped classroom optics curriculum. Resident OKAP examination performance for the 4 years before and after changing the optics curriculum from a lecture-based to flipped classroom curriculum (mean ± 95% CI). See [Table 2] for full-length OKAP subsection titles. Asterisk symbol denotes statistically significant difference at p = 0.00008. CI, confidence interval.

Discussion

To date, there has been minimal research into the application of alternative educational methodologies such as the flipped classroom format to ophthalmology education.[11] [12] [13] Our data show that transitioning from traditional lectures to a flipped classroom curriculum for clinical optics was associated with a statistically significant increase in mean OKAP examination score for the Optics, Refraction, and Contact Lens subsection. This suggests that the flipped classroom format may be more effective than lectures for facilitating ophthalmology resident clinical optics knowledge and ability to apply this knowledge in a standardized test setting. Standardized test scores are relevant, since a written qualifying examination (a standardized test similar in nature to the OKAP examination) is required for American Board of Ophthalmology certification, and the board pass rate for graduates is in turn a key metric monitored by the Accreditation Council for Graduate Medical Education (ACGME) ophthalmology residency review committee for the purpose of residency program accreditation.[14] [15]

The flipped classroom format as a form of active learning has been shown to positively influence learner engagement, motivation, and satisfaction.[4] [16] [17] [18] [19] Our flipped classroom optics curriculum places learners in a more active role, requiring them to both complete problem sets independently and teach their colleagues. Of note, our learners were also given a choice of reference materials to utilize during the pre-class self-study interval. Trainee learning preferences may differ, with some favoring prerecorded video lectures and others preferring reference books. While the average in class time dedicated to our flipped classroom curriculum was less than that of our lecture-based curriculum, flipped classroom learners were required to spend additional time outside the classroom to complete problem sets. We did not quantify the number of hours spent by residents outside of the classroom, but based on our own experience completing the problem sets, we estimate the total time spent by learners in and out of the classroom to be roughly similar between the two curricular approaches. Feedback from learners suggests that the average time spent completing the self-study portion of the flipped classroom curriculum decreases with each year of experience.

Interestingly, PGY-2 and PGY-3 but not PGY-4 residents exposed to our flipped classroom curriculum performed significantly better on the OKAP examination than respective PGY counterparts exposed to a lecture-based optics curriculum. This data suggests that the flipped classroom format benefited inexperienced optics learners more so than those with 2 years of pre-existing optics examination experience. Of course, subgroup analysis is more prone to type 1 statistical error given the smaller sample size.

Anecdotally, we found our problem-based optics curriculum to be less burdensome and more enjoyable for instructors. Faculty were no longer required to spend time generating slide presentations for the optics lecture series. Providing faculty with problem set answer keys allowed them to efficiently prepare for in-class sessions. Additionally, faculty time commitment required for in-class educational sessions was less with the flipped classroom curriculum than with traditional lectures. While a similar group of optometry and ophthalmology teaching faculty was utilized in this study both before and after the curriculum change, we did find that the flipped classroom format with pre-constructed questions and answer key was more conducive to the recruitment of faculty instructors. The flipped classroom format likely lowers the barrier to participation, especially for faculty without a clinical practice that focuses on optics. This may be helpful for residency training programs that struggle to recruit faculty to teach optics.

This study is subject to several limitations including those associated with a retrospective, nonrandomized study design based on data from a single institution. A limitation of our study involves the sampling error that can occur due to the relatively few questions on any one OKAP examination subsection. As noted in the OKAP user guide: “because each yearly exam is comprised of a different set of test items, not every clinical point can be covered in every year. […] Because subtests necessarily must contain fewer items than the test as a whole, subtest scores are somewhat less reliable and must be interpreted with more caution.”[20]

Another limitation of our study is that we are unable to separate the contributions of the different components of our flipped classroom curriculum. The required self-study problem set, learners as teachers small group discussion format, and the timing of curriculum prior to testing were all factors that may have contributed to the performance improvement observed.

While we do not have data to directly test whether the required problem set alone would be sufficient to improve optics scores, there is extensive literature suggesting that collaborative learning yields superior educational outcomes relative to individualistic learning.[19] Additionally, our data suggests that simply providing or improving access to problem sets is not sufficient to optimize examination performance. OKAP-style practice questions were made available to all residents throughout the study period, including those found in review questions in ophthalmology and multiple editions of the BCSC series.[9] [10] Also of note, ophthoquestions.com, an online resource that could potentially increase learner access to OKAP-style practice questions, was introduced in September 2011. Our program has never paid for this online question bank for our trainees and we do not track those that utilize it; however, ophthoquestions.com company representatives report that 95% of ophthalmology trainees utilize their services for OKAP preparation.[21] No statistically significant difference was identified when comparing optics subsection scores before and after the introduction of ophthoquestions.com (and before the introduction of the flipped classroom curriculum). Similarly, no statistically significant difference in nonoptics subsection test scores was identified when comparing scores prior to and after the introduction of ophthoquestions.com. This suggests that improved online access to optics questions as well as nonoptics questions does not significantly improve OKAP scores.

Our dataset is not sufficient to fully evaluate whether the temporal shift of optics curriculum to more closely precede the OKAP test (delivered in March) was influential, but suggests that the effect, if present, was small. There was no statistically significant difference between mean subsection exam scores for curricula presented in the fall versus late winter (51.7 ± 2.0 and 52.8 ± 2.3, respectively, mean ± 95% CI; p = 0.23).

While our research shows that the flipped classroom format can be applied in an advantageous manner to improve ophthalmology resident optics standardized test scores, further study will be needed to determine whether this curricular shift translates to an improved ability to apply optics knowledge to real-world clinical practice. And finally, while these results may be applicable to the optics portion of the ophthalmology residency curriculum, it is unclear whether flipped classroom would also be more effective than traditional lectures for other ophthalmologic areas of study.

Despite these limitations, given the relative advantages of flipped classroom optics learning over traditional lecture-based curricula, a flipped classroom approach is a reasonable option for ophthalmology training programs seeking to improve resident performance in clinical optics.

Conflict of Interest

None declared.

• References

• 1 Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ 2018; 18 (01) 38
• 2 Rao SP, DiCarlo SE. Peer instruction improves performance on quizzes. Adv Physiol Educ 2000; 24 (01) 51-55
• 3 Gillette C, Rudolph M, Kimble C, Rockich-Winston N, Smith L, Broedel-Zaugg K. A meta-analysis of outcomes comparing flipped classroom and lecture. Am J Pharm Educ 2018; 82 (05) 6898
• 4 Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ 2017; 51 (06) 585-597
• 5 Riddell J, Jhun P, Fung CC. , et al. Does the flipped classroom improve learning in graduate medical education?. J Grad Med Educ 2017; 9 (04) 491-496
• 6 Lancaster WB. Physiologic optics the basis for teaching clinical ophthalmology. JAMA 1913; 61 (13) 5
• 7 Guyton DL, West CE, Miller JM. , et al. Ophthalmic Optics and Clinical Refraction . Baltimore, MD: Prism Press; . 2nd edition; 1999
• 8 Hunter DG. Optics & Refraction Lecture Series. 2010; http://www.lastminuteoptics.com/
• 9 Basic and clinical science course section 3: Clinical optics Vol 3. San Francisco, CA: American Academy of Ophthalmology; 2008–2015
• 10 Chern KC, Wright KW. Review Questions in Ophthalmology. 2nd edition. Philadelphia, PA: Lippincott Williams & Wilkins; 2004
• 11 Al Wadani F, Khan AR. Problem-based learning in ophthalmology: a brief review. Oman J Ophthalmol 2014; 7 (01) 1-2
• 12 Tang F, Chen C, Zhu Y. , et al. Comparison between flipped classroom and lecture-based classroom in ophthalmology clerkship. Med Educ Online 2017; 22 (01) 1395679
• 13 Lin Y, Zhu Y, Chen C. , et al. Facing the challenges in ophthalmology clerkship teaching: is flipped classroom the answer?. PLoS One 2017; 12 (04) e0174829
• 14 American Board of Ophthalmology. https://abop.org/
• 15 Accreditation Council for Graduate Medical Education. https://www.acgme.org/Specialties/Documents-and-Resources/pfcatid/13/Ophthalmology
• 16 Chen KS, Monrouxe L, Lu YH. , et al. Academic outcomes of flipped classroom learning: a meta-analysis. Med Educ 2018
• 17 McLaughlin JE, Roth MT, Glatt DM. , et al. The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Acad Med 2014; 89 (02) 236-243
• 18 Evans KH, Thompson AC, O'Brien C. , et al. An innovative blended preclinical curriculum in clinical epidemiology and biostatistics: impact on student satisfaction and performance. Acad Med 2016; 91 (05) 696-700
• 19 Prince M. Does active learning work? A review of the research. J Eng Educ 2004; 93 (03) 223-231
• 20 American Academy of Ophthalmology. OKAP user's guide. https://www.aao.org/okap-exam
• 21 OphthoQuestions. https://www.ophthoquestions.com/