Keywords curriculum - needs assessment - resident - technology
The Accreditation Council for Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in Ophthalmology delineates standards and expectations for the effective training of ophthalmology residents. Residents must be educated in basic and clinical sciences via structured didactic sessions, including formal teaching case presentation conferences. As part of this process, residents are expected to use information technology to optimize learning (a common program requirement). The use of information technology in education often takes the form of e-learning: utilizing electronic technology to access educational curriculum outside of a traditional classroom. Blended learning, face-to-face teaching plus e-learning, produces superior results in student knowledge, confidence, and satisfaction.[1 ]
[2 ] E-learning elements are more effective when presented in a unified learning management system.[3 ] The use of e-learning and blended learning in nursing education has been well-described.[4 ]
[5 ]
[6 ]
[7 ]
[8 ]
[9 ]
[10 ]
[11 ] E-learning has also been utilized successfully in resident education in some subspecialties such as obstetrics and gynecology,[12 ] otolaryngology,[13 ] emergency medicine,[14 ] psychiatry,[15 ] and orthopedic surgery.[16 ] The current body of literature, however, provides relatively little information regarding the use of information technology in teaching and learning in ophthalmology residency training. In particular, very little is known regarding ophthalmology residents' and faculty members' use of and attitude toward technology in teaching, including blended learning and e-learning. The goals of this study were to conduct a Web-based survey to evaluate the current state of technology in teaching and learning in our ophthalmology residency program, and to use these data for program improvement. We report ophthalmology residents' and faculty members' attitudes toward and potential barriers to use of technology that may be used to design curriculum to improve resident training in ophthalmology.
Methods
We developed two Web-based, cross-sectional, voluntary surveys (one for residents and one for faculty members) to assess the current use of technology in teaching and learning in the ophthalmology training program. The main objective was to guide future curriculum development/revision.
Survey items were developed around three major themes for residents: (1) learning style, (2) current use of technology, and (3) methods of incorporating technology that were perceived as beneficial or desired. For faculty, items focused on: (1) current use of technology, (2) methods of incorporating technology that were perceived as beneficial or desired, and (3) barriers preventing use of technology. The surveys consisted of 5-point Likert scales (resident learning style) or multiple-choice formats, with some questions requiring one answer and others allowing for the selection of multiple answers. Open text boxes were included to capture feedback not specified in the survey. Responses were collected anonymously. The study was approved by the Vanderbilt University Medical Center Institutional Review Board 161969.
Data Collection and Analysis
In October 2016, we e-mailed the survey request with the respective survey link to current residents (N = 14) and faculty members who teach resident didactic sessions (N = 38). Reminder e-mails were sent twice to maximize survey responses at 2 and 4 weeks following the initial mailing. Because respondents were anonymous, all residents and teaching faculty members were sent reminder e-mails. Data were stored in a secure location in the corresponding author's office within a secure survey account. Descriptive statistics, including frequency, percentage, mean, and standard deviation, were calculated to determine the characteristics of respondents and their responses to survey questions. To understand if there was a difference in learning styles of residents, we calculated six Wilcoxon signed–rank tests. Because the tests were predetermined and few, we did not correct for multiple comparison.
Results
Resident Survey
Our response rate was 100% (14 of 14).
Learning Style
Residents ranked themselves with respect to learning style on a 5-point rating scale from 1 (I do not learn well this way) to 5 (This is my primary learning style). Visual was rated as the primary learning style by 8 of 14 (57.1%) residents and kinesthetic by 5 of 14 (35.7%). Only 1 resident (7.1%) rated reading/writing as a primary learning style and no resident rated auditory as primary ([Table 1 ]).
Table 1
Resident learning style
Learning Style
Mean (1 to 5)
Median
Mode
Standard deviation
Range
Visual
4.5
5
5
0.650
3 to 5
Reading/Writing
3.36
3
3
0.745
2 to 5
Auditory
2.64
2.5
2
0.929
1 to 4
Kinesthetic
4.14
4
4
0.864
3 to 5
We found differences between all of the comparisons at the p -value of 0.05 level except for visual compared with kinesthetic learning style. [Fig. 1 ] compares the distributions of the four learning styles.
Fig. 1 Comparison of resident's style of learning.
[Table 2 ] illustrates 95% confidence intervals for the comparisons of learning styles. With 95% confidence interval, we expect residents to rank the visual learning style between 1 and 2 points higher versus the read/write style. Similarly, we expect them to rank between 1.5 and 2.5 points higher for visual versus auditory styles. The other comparisons may be interpreted similarly. There was no statistically significant difference between the visual and kinesthetic learning styles.
Table 2
Learning style comparison
Confidence interval
95% lower CL
95% upper CL
p -Value
Visual/Read.Write
1.00
2.00
0.01
Visual/Auditory
1.50
2.50
0.00
Visual/Kinesthetic
–1.00
2.00
0.28
Read.Write/Auditory
0.00
2.00
0.03
Read.Write/Kinesthetic
–2.00
–0.50
0.03
Auditory/Kinesthetic
–2.50
–1.00
0.01
Abbreviation: CL, confidence level.
Current Use of Technology
Most residents used information technology to answer questions in online quizzes or question banks (n = 12, 85.7%) and watch videos (n = 9, 64.3%). Listening to podcasts (n = 3, 21.4%) and playing learning games (n = 3, 21.4%) were less preferred methods of using technology. Watching recorded lectures (n = 1, 7.1%) was entered as a free-text response ([Fig. 2 ]).
Fig. 2 Residents' current use of technology.
Desired Methods of Incorporating Technology
When queried regarding what new technology would be beneficial to incorporate, a majority of residents affirmed that the curriculum would benefit from a Web platform for all curriculum components (schedule, assignments, supplemental materials) in one place (n = 9, 64.3%), an app for faculty to provide instant feedback on surgical performance (n = 7, 50%), and videos to supplement lecture (n = 7, 50%). A Web platform to connect with researchers in the department (n = 6, 42.9%), prerecorded, on-demand white board sessions (n = 5, 35.7%), and podcasts (n = 4, 28.6%) were less preferred methods of incorporating technology. Least preferred methods included online quizzes (n = 1, 7.1%), an app to enable residents give faculty instant feedback on lectures (n = 1, 7.1%), and other (n = 2, 14.2%). Free-text entries included access to all lecture slides online (n = 1, 7.1%) and increased availability of wet laboratory resources (n = 1, 7.1%) ([Fig. 3 ]).
Fig. 3 Residents' preferred methods to incorporate technology.
Faculty Survey
Current Use of Technology
Among teaching faculty members who completed the survey (response rate 26 of 38, 68%), 5 (19.2%) “always,” 12 (46.2%) “sometimes,” and 9 (34.6%) “never” used technology (audience response clickers, flipped classroom, online quizzes, videos, etc.) to supplement their lectures. The most commonly used technology included online pre- or postquizzes (n = 12, 46.2%), audience response systems/clickers (n = 10, 38.5%), YouTube or other videos (n = 9, 34.6%), and flipped classroom (n = 8, 30.8%).
Desired Methods of Incorporating Technology
A majority of faculty members affirmed that the resident curriculum would benefit from a Web platform for all curriculum components (schedule, assignments, supplemental materials) in one place (n = 19, 73.1%), a Web platform for residents and clinical faculty to connect with researchers in the department (n = 17, 65.4%), online quizzes (n = 16, 61.5%), and videos to supplement lecture (n = 15, 57.7%). Less preferred methods included an app for residents to provide feedback to faculty on lectures (n = 12, 46.2%), an app for faculty to give residents instant feedback on surgical performance (n = 10, 38.5%), and prerecorded, on-demand white board sessions (n = 6, 23.1%) ([Fig. 4 ]).
Fig. 4 Faculty respondents' preferred methods to incorporate technology.
Barriers to Faculty Use of Technology
Commonly cited barriers to incorporating new technology into teaching included lack of knowledge of available technology (n = 15, 57.7%), lack of time to change existing lectures/power points (n = 13, 50%), and lack of knowledge of how to use technology (n = 12, 46.2%). Three faculty respondents (11.5%) reported that they did not think technology is useful in teaching. Free-text entries indicated respondents' desire for departmental support and education regarding use of technology in teaching: “I think we need updates during normal business hours where we can find out what technology is available and see it demonstrated. Alternatively, this could be done one-on-one with interested/involved faculty .” “Administrative/IT help in preparing lectures… ” “I believe that implementing new teaching technology, multimedia and teaching archives has multiple benefits that outweigh concerns for infrastructure costs .” The comments indicate an additional perceived barrier of lack of administrative and financial support for adoption of new didactic methods ([Fig. 5 ]).
Fig. 5 Faculty respondents' identified barriers to incorporating technology.
Discussion
We examined ophthalmology resident and faculty attitudes toward the use of technology in learning and teaching to address the ACGME call for use of technology in teaching. We focused on three themes for residents: learning style, current use of technology in learning, and potentially beneficial uses of technology in the resident curriculum. For faculty, we focused on current use of technology in teaching, potentially beneficial uses of technology in the resident curriculum, and barriers to incorporating technology into teaching.
Both resident and faculty respondents indicated that a central Web platform for all curriculum components would benefit the curriculum, as would a Website to connect with researchers in the department. Both groups also noted that videos to supplement lecture would be beneficial. Feedback was a third area of agreement between the groups. Interestingly, each group of respondents valued an app to receive feedback (residents for surgical performance and faculty for lectures) more than an app to give feedback. This may be attributable to the extra time and effort that would be required to provide feedback via an app as compared with the relative ease and benefit of receiving rapid, standardized feedback.
Faculty and residents did not agree on the value of online quizzes. A majority of faculty respondents, as compared with only one resident respondent, felt that online quizzes would benefit the curriculum. This may reflect the fact that residents are already using online quizzes and question banks and therefore do not see them as something that needs to be added. Faculty respondents also listed online quizzes among the types of technology in teaching they currently use, perhaps indicating a preference for modalities with which they are already comfortable. Faculty disagreed regarding the usefulness of technology in teaching: several respondents expressed strong value for technology, whereas a minority felt it was not useful.
Residents preferred visual and kinesthetic learning over reading/writing and auditory. Residents' preference for visual learning was reflected in their use of technology to watch videos. Visual and kinesthetic learning preference argues against the traditional model of reading and lecture-based knowledge acquisition in residency. Teaching modalities that capitalize on the strengths and preferences of the learners may enhance both retention and enjoyment of the curriculum. Awareness of the preferred learning style of residents can guide the creation of tailored e-learning resources.
Limitations of this study include a small sample size from a single institution. Demographic characteristics masked by anonymous survey responses may have introduced bias. For example, it is possible that faculty who elected to respond to an online survey represent a subset who are already comfortable with use of technology. Strengths of the study include our excellent response rate (100%) for residents and faculty (68%). We gained insight into current use of technology by both residents and faculty, and which modalities each group deemed useful and worth incorporating, despite perceived barriers. Such insight allows curriculum development to be tailored to the needs of the learners and faculty development to the needs of the teachers.
It may be useful to expand this type of survey to a larger group of respondents, such as residents and faculty from other ophthalmology training programs. Use of similar surveys in other specialty areas could elucidate differences in residents' learning style and preferences among specialties as well as faculty attitudes toward, and barriers to use of, technology in teaching. However, each program has unique resources, depth of faculty, number of residents, and attitudes toward technology. It is reasonable, therefore, to begin locally prior to more broad-based surveys.
Conclusion
We conducted a systematic needs assessment of ophthalmology residents and faculty and identified resident preferences for visual and/or kinesthetic learning styles as well as respondents' values related to e-learning. Our results were limited to one residency program but highlight issues that may be common to many institutions. Among these are potential disconnect between faculty and learners regarding beneficial teaching methods, as well as likely barriers among faculty to implementation of new teaching formats involving greater use of technology. Our study is an example of a best practice for exploring these issues in an individual program and demonstrates how resident and faculty responses can provide valuable information to create a framework for curriculum development and program improvement. Further study is needed to determine the most effective ways to use technology in resident education. Comparison of resident and faculty responses can help bridge the gap between learner expectations and faculty preferences as residency programs embark on curriculum revision and inclusion of technology.
