Technology Use in Ophthalmology Resident Education: Results to Aid Program Directors in Curriculum Development

Abstract

Background The Accreditation Council for Graduate Medical Education in ophthalmology lists the use of information technology to optimize learning as a common program requirement. The use of technology in published studies often takes the form of e-learning. No study to date outlines what methods are preferred by residents and faculty in United States Ophthalmology programs.

Objective We conducted a needs assessment to evaluate the current state of technology in teaching and learning in an ophthalmology residency program, to identify barriers to using technology, and to determine areas for future curriculum development.

Methods We used an anonymous online survey to assess current residents and faculty within our ophthalmology residency program.

Results Residents identified their primary learning styles as visual and kinesthetic and they preferred videos and online question banks.

More than 35% of faculty respondents “never” use technology in teaching. Among faculty who do use technology, online quizzes and videos were the most common modalities used to supplement lectures. Common barriers to incorporating technology included lack of time, lack of knowledge of available technologic tools, and lack of skill in using technology.

Both faculty and residents identified a Web platform for curricular elements and a feedback app as potentially beneficial additions to the curriculum. Each group rated an app to receive feedback more highly than an app to provide it. The two groups disagreed on the usefulness of online quizzes. There was variability among faculty responses regarding the usefulness of technology in teaching.

Conclusion Our needs assessment identified areas of agreement among residents and faculty as well as mismatches and barriers regarding the use of technology to support learning by ophthalmology residents. Our study is an example of a best practice for use of a needs assessment to provide a framework for curriculum development and program improvement in an individual program. Further research is needed to generalize these results across ophthalmology programs.

Keywords

curriculum - needs assessment - resident - technology

The Accreditation Council for Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in Ophthalmology delineates standards and expectations for the effective training of ophthalmology residents. Residents must be educated in basic and clinical sciences via structured didactic sessions, including formal teaching case presentation conferences. As part of this process, residents are expected to use information technology to optimize learning (a common program requirement). The use of information technology in education often takes the form of e-learning: utilizing electronic technology to access educational curriculum outside of a traditional classroom. Blended learning, face-to-face teaching plus e-learning, produces superior results in student knowledge, confidence, and satisfaction.[1] [2] E-learning elements are more effective when presented in a unified learning management system.[3] The use of e-learning and blended learning in nursing education has been well-described.[4] [5] [6] [7] [8] [9] [10] [11] E-learning has also been utilized successfully in resident education in some subspecialties such as obstetrics and gynecology,[12] otolaryngology,[13] emergency medicine,[14] psychiatry,[15] and orthopedic surgery.[16] The current body of literature, however, provides relatively little information regarding the use of information technology in teaching and learning in ophthalmology residency training. In particular, very little is known regarding ophthalmology residents' and faculty members' use of and attitude toward technology in teaching, including blended learning and e-learning. The goals of this study were to conduct a Web-based survey to evaluate the current state of technology in teaching and learning in our ophthalmology residency program, and to use these data for program improvement. We report ophthalmology residents' and faculty members' attitudes toward and potential barriers to use of technology that may be used to design curriculum to improve resident training in ophthalmology.

Methods

We developed two Web-based, cross-sectional, voluntary surveys (one for residents and one for faculty members) to assess the current use of technology in teaching and learning in the ophthalmology training program. The main objective was to guide future curriculum development/revision.

Survey items were developed around three major themes for residents: (1) learning style, (2) current use of technology, and (3) methods of incorporating technology that were perceived as beneficial or desired. For faculty, items focused on: (1) current use of technology, (2) methods of incorporating technology that were perceived as beneficial or desired, and (3) barriers preventing use of technology. The surveys consisted of 5-point Likert scales (resident learning style) or multiple-choice formats, with some questions requiring one answer and others allowing for the selection of multiple answers. Open text boxes were included to capture feedback not specified in the survey. Responses were collected anonymously. The study was approved by the Vanderbilt University Medical Center Institutional Review Board 161969.

Data Collection and Analysis

In October 2016, we e-mailed the survey request with the respective survey link to current residents (N = 14) and faculty members who teach resident didactic sessions (N = 38). Reminder e-mails were sent twice to maximize survey responses at 2 and 4 weeks following the initial mailing. Because respondents were anonymous, all residents and teaching faculty members were sent reminder e-mails. Data were stored in a secure location in the corresponding author's office within a secure survey account. Descriptive statistics, including frequency, percentage, mean, and standard deviation, were calculated to determine the characteristics of respondents and their responses to survey questions. To understand if there was a difference in learning styles of residents, we calculated six Wilcoxon signed–rank tests. Because the tests were predetermined and few, we did not correct for multiple comparison.

Results

Resident Survey

Our response rate was 100% (14 of 14).

Learning Style

Residents ranked themselves with respect to learning style on a 5-point rating scale from 1 (I do not learn well this way) to 5 (This is my primary learning style). Visual was rated as the primary learning style by 8 of 14 (57.1%) residents and kinesthetic by 5 of 14 (35.7%). Only 1 resident (7.1%) rated reading/writing as a primary learning style and no resident rated auditory as primary ([Table 1]).

Table 1 Resident learning style

Learning Style	Mean (1 to 5)	Median	Mode	Standard deviation	Range
Visual	4.5	5	5	0.650	3 to 5
Reading/Writing	3.36	3	3	0.745	2 to 5
Auditory	2.64	2.5	2	0.929	1 to 4
Kinesthetic	4.14	4	4	0.864	3 to 5

We found differences between all of the comparisons at the p-value of 0.05 level except for visual compared with kinesthetic learning style. [Fig. 1] compares the distributions of the four learning styles.

Fig. 1 Comparison of resident's style of learning.

[Table 2] illustrates 95% confidence intervals for the comparisons of learning styles. With 95% confidence interval, we expect residents to rank the visual learning style between 1 and 2 points higher versus the read/write style. Similarly, we expect them to rank between 1.5 and 2.5 points higher for visual versus auditory styles. The other comparisons may be interpreted similarly. There was no statistically significant difference between the visual and kinesthetic learning styles.

Table 2 Learning style comparison

Confidence interval	95% lower CL	95% upper CL	p-Value
Visual/Read.Write	1.00	2.00	0.01
Visual/Auditory	1.50	2.50	0.00
Visual/Kinesthetic	–1.00	2.00	0.28
Read.Write/Auditory	0.00	2.00	0.03
Read.Write/Kinesthetic	–2.00	–0.50	0.03
Auditory/Kinesthetic	–2.50	–1.00	0.01

Abbreviation: CL, confidence level.

Current Use of Technology

Most residents used information technology to answer questions in online quizzes or question banks (n = 12, 85.7%) and watch videos (n = 9, 64.3%). Listening to podcasts (n = 3, 21.4%) and playing learning games (n = 3, 21.4%) were less preferred methods of using technology. Watching recorded lectures (n = 1, 7.1%) was entered as a free-text response ([Fig. 2]).

Fig. 2 Residents' current use of technology.

Desired Methods of Incorporating Technology

When queried regarding what new technology would be beneficial to incorporate, a majority of residents affirmed that the curriculum would benefit from a Web platform for all curriculum components (schedule, assignments, supplemental materials) in one place (n = 9, 64.3%), an app for faculty to provide instant feedback on surgical performance (n = 7, 50%), and videos to supplement lecture (n = 7, 50%). A Web platform to connect with researchers in the department (n = 6, 42.9%), prerecorded, on-demand white board sessions (n = 5, 35.7%), and podcasts (n = 4, 28.6%) were less preferred methods of incorporating technology. Least preferred methods included online quizzes (n = 1, 7.1%), an app to enable residents give faculty instant feedback on lectures (n = 1, 7.1%), and other (n = 2, 14.2%). Free-text entries included access to all lecture slides online (n = 1, 7.1%) and increased availability of wet laboratory resources (n = 1, 7.1%) ([Fig. 3]).

Fig. 3 Residents' preferred methods to incorporate technology.

Faculty Survey

Current Use of Technology

Among teaching faculty members who completed the survey (response rate 26 of 38, 68%), 5 (19.2%) “always,” 12 (46.2%) “sometimes,” and 9 (34.6%) “never” used technology (audience response clickers, flipped classroom, online quizzes, videos, etc.) to supplement their lectures. The most commonly used technology included online pre- or postquizzes (n = 12, 46.2%), audience response systems/clickers (n = 10, 38.5%), YouTube or other videos (n = 9, 34.6%), and flipped classroom (n = 8, 30.8%).

Desired Methods of Incorporating Technology

A majority of faculty members affirmed that the resident curriculum would benefit from a Web platform for all curriculum components (schedule, assignments, supplemental materials) in one place (n = 19, 73.1%), a Web platform for residents and clinical faculty to connect with researchers in the department (n = 17, 65.4%), online quizzes (n = 16, 61.5%), and videos to supplement lecture (n = 15, 57.7%). Less preferred methods included an app for residents to provide feedback to faculty on lectures (n = 12, 46.2%), an app for faculty to give residents instant feedback on surgical performance (n = 10, 38.5%), and prerecorded, on-demand white board sessions (n = 6, 23.1%) ([Fig. 4]).

Fig. 4 Faculty respondents' preferred methods to incorporate technology.

Barriers to Faculty Use of Technology

Commonly cited barriers to incorporating new technology into teaching included lack of knowledge of available technology (n = 15, 57.7%), lack of time to change existing lectures/power points (n = 13, 50%), and lack of knowledge of how to use technology (n = 12, 46.2%). Three faculty respondents (11.5%) reported that they did not think technology is useful in teaching. Free-text entries indicated respondents' desire for departmental support and education regarding use of technology in teaching: “I think we need updates during normal business hours where we can find out what technology is available and see it demonstrated. Alternatively, this could be done one-on-one with interested/involved faculty.” “Administrative/IT help in preparing lectures…” “I believe that implementing new teaching technology, multimedia and teaching archives has multiple benefits that outweigh concerns for infrastructure costs.” The comments indicate an additional perceived barrier of lack of administrative and financial support for adoption of new didactic methods ([Fig. 5]).

Fig. 5 Faculty respondents' identified barriers to incorporating technology.

Discussion

We examined ophthalmology resident and faculty attitudes toward the use of technology in learning and teaching to address the ACGME call for use of technology in teaching. We focused on three themes for residents: learning style, current use of technology in learning, and potentially beneficial uses of technology in the resident curriculum. For faculty, we focused on current use of technology in teaching, potentially beneficial uses of technology in the resident curriculum, and barriers to incorporating technology into teaching.

Both resident and faculty respondents indicated that a central Web platform for all curriculum components would benefit the curriculum, as would a Website to connect with researchers in the department. Both groups also noted that videos to supplement lecture would be beneficial. Feedback was a third area of agreement between the groups. Interestingly, each group of respondents valued an app to receive feedback (residents for surgical performance and faculty for lectures) more than an app to give feedback. This may be attributable to the extra time and effort that would be required to provide feedback via an app as compared with the relative ease and benefit of receiving rapid, standardized feedback.

Faculty and residents did not agree on the value of online quizzes. A majority of faculty respondents, as compared with only one resident respondent, felt that online quizzes would benefit the curriculum. This may reflect the fact that residents are already using online quizzes and question banks and therefore do not see them as something that needs to be added. Faculty respondents also listed online quizzes among the types of technology in teaching they currently use, perhaps indicating a preference for modalities with which they are already comfortable. Faculty disagreed regarding the usefulness of technology in teaching: several respondents expressed strong value for technology, whereas a minority felt it was not useful.

Residents preferred visual and kinesthetic learning over reading/writing and auditory. Residents' preference for visual learning was reflected in their use of technology to watch videos. Visual and kinesthetic learning preference argues against the traditional model of reading and lecture-based knowledge acquisition in residency. Teaching modalities that capitalize on the strengths and preferences of the learners may enhance both retention and enjoyment of the curriculum. Awareness of the preferred learning style of residents can guide the creation of tailored e-learning resources.

Limitations of this study include a small sample size from a single institution. Demographic characteristics masked by anonymous survey responses may have introduced bias. For example, it is possible that faculty who elected to respond to an online survey represent a subset who are already comfortable with use of technology. Strengths of the study include our excellent response rate (100%) for residents and faculty (68%). We gained insight into current use of technology by both residents and faculty, and which modalities each group deemed useful and worth incorporating, despite perceived barriers. Such insight allows curriculum development to be tailored to the needs of the learners and faculty development to the needs of the teachers.

It may be useful to expand this type of survey to a larger group of respondents, such as residents and faculty from other ophthalmology training programs. Use of similar surveys in other specialty areas could elucidate differences in residents' learning style and preferences among specialties as well as faculty attitudes toward, and barriers to use of, technology in teaching. However, each program has unique resources, depth of faculty, number of residents, and attitudes toward technology. It is reasonable, therefore, to begin locally prior to more broad-based surveys.

Conclusion

We conducted a systematic needs assessment of ophthalmology residents and faculty and identified resident preferences for visual and/or kinesthetic learning styles as well as respondents' values related to e-learning. Our results were limited to one residency program but highlight issues that may be common to many institutions. Among these are potential disconnect between faculty and learners regarding beneficial teaching methods, as well as likely barriers among faculty to implementation of new teaching formats involving greater use of technology. Our study is an example of a best practice for exploring these issues in an individual program and demonstrates how resident and faculty responses can provide valuable information to create a framework for curriculum development and program improvement. Further study is needed to determine the most effective ways to use technology in resident education. Comparison of resident and faculty responses can help bridge the gap between learner expectations and faculty preferences as residency programs embark on curriculum revision and inclusion of technology.

Conflict of Interest

None declared.

Acknowledgements

This work was produced with the support of the Educator Development Program within the Office for Continuous Professional Development/Office of Health Sciences Education, Vanderbilt University School of Medicine/Vanderbilt University Medical Center.

Portions of this research were presented as a poster at the Gerald S. Gotterer Health Professions Education Research Day (HPERD), Vanderbilt University School of Medicine, December 9, 2016.

• References

• 1 Beattie J, Brady L, Tobias T. Improving clinician confidence and skills: piloting a web-based learning program for clinicians in supportive care screening of cancer patients. J Cancer Educ 2014; 29 (01) 38-43
• 2 Lindeman BMLJ, Law JK, Lipsett PA, Arbella T, Stem M, Lidor AO. A blended online curriculum in the basic surgery clerkship: a pilot study. Am J Surg 2015; 209 (01) 145-151
• 3 Wagner JPCD, Chen DC, Donahue TR. , et al. Assessment of resident operative performance using a real-time mobile Web system: preparing for the milestone age. J Surg Educ 2014; 71 (06) e41-e46
• 4 Hsu LL, Hsieh SI. Factors affecting metacognition of undergraduate nursing students in a blended learning environment. Int J Nurs Pract 2014; 20 (03) 233-241
• 5 Lahti M, Kontio R, Pitkänen A, Välimäki M. Knowledge transfer from an e-learning course to clinical practice. Nurse Educ Today 2014; 34 (05) 842-847
• 6 Liaw SY, Wong LF, Chan SW. , et al. Designing and evaluating an interactive multimedia Web-based simulation for developing nurses' competencies in acute nursing care: randomized controlled trial. J Med Internet Res 2015; 17 (01) e5
• 7 McCutcheon K, Lohan M, Traynor M, Martin D. A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. J Adv Nurs 2015; 71 (02) 255-270
• 8 Mettiäinen S, Luojus K, Salminen S, Koivula M. Web course on medication administration strengthens nursing students' competence prior to graduation. Nurse Educ Pract 2014; 14 (04) 368-373
• 9 Pucer P, Trobec I, Žvanut B. An information communication technology based approach for the acquisition of critical thinking skills. Nurse Educ Today 2014; 34 (06) 964-970
• 10 Schnetter VA, Lacy D, Jones MM, Bakrim K, Allen PE, O'Neal C. Course development for web-based nursing education programs. Nurse Educ Pract 2014; 14 (06) 635-640
• 11 Sowan AK, Idhail JA. Evaluation of an interactive web-based nursing course with streaming videos for medication administration skills. Int J Med Inform 2014; 83 (08) 592-600
• 12 Ghareeb A, Han H, Delfino K, Taylor F. Blended learning in obstetrics and gynecology resident education: impact on resident clinical performance. J Med Educ Curric Dev 2016; 3: pii : JMECD.S40598
• 13 Tarpada SP, Hsueh WD, Gibber MJ. Resident and student education in otolaryngology: a 10-year update on e-learning. Laryngoscope 2017; 127 (07) E219-E224
• 14 Dankbaar ME, Roozeboom MB, Oprins EA. , et al. Preparing residents effectively in emergency skills training with a serious game. Simul Healthc 2017; 12 (01) 9-16
• 15 Kuhn E, Hugo E. Technology-based blended learning to facilitate psychiatry resident training in prolonged exposure (PE) therapy for PTSD. Acad Psychiatry 2017; 41 (01) 121-124
• 16 Tarpada SP, Morris MT, Burton DA. E-learning in orthopedic surgery training: a systematic review. J Orthop 2016; 13 (04) 425-430