Keywords
Online learning platform - gamification - hybrid learning routine in radiology - preparation
and follow-up work - Just-in-Time Teaching
Introduction
The measures put into place in 2020 in response to the corona virus required rapid
implementation of fully digital teaching, specifically in the daily university routine,
due to the massive restrictions on in-person classes. With the easing of restrictions,
in-person teaching at universities became increasingly established again. However,
only 14% of students prefer a learning model consisting of only in-person teaching,
while 86% prefer a hybrid learning model [1]. Digital media are readily available and provide flexibility and a cooperative and
communicative learning arrangement for student and professional training [2]. These known didactic advantages of a hybrid learning concept should also be integrated
in didactics in medicine, which experienced a fundamental boost during the pandemic
[3].
Due to the wide range of imaging methods, radiology is suitable for such digital integration.
Moreover, due to the relevance of radiology for all disciplines, learning success
in this area should be maximized to ensure optimal learning of the important principles
of imaging. Radiology is extremely important because of its synergy with other disciplines.
Since, in our experience, it is difficult to teach all content to be learned during
the student training period, we feel that there is a need for new and innovative methods
of learning. According to Miller et al., the main responsibility of medical students
is to learn medicine [4]. Therefore, medical teaching should allow need-based learning.
As a result, we developed the online learning platform Raditorium.com to allow hybrid teaching and to be used for our core radiology concepts [5].
Raditorium.com is a website that allows students to perform digital preparation and follow-up of
radiology concepts by providing handouts, case examples, self-tests, and lecture slides
for a particular course (Supp. 1). An analogy can be drawn between this teaching concept and the training concept
used in high-performance sports. An athlete goes through different phases of preparation
and follow-up when training for a competition in order to achieve optimal performance
during the competition and to increase performance in the long term. The use of preparation
and follow-up can be identified as a key factor for later success and in the case
of teaching allows access to adapted learning content. Raditorium.com offers students flexible access to simple but essential preparation and follow-up,
thereby increasing the motivation for and effectiveness of each in-person class.
In the case of digital learning, Reinmann differentiates between e-learning by distributing
(distribution), e-learning by interacting (interaction), and e-learning by collaborating
(collaboration) and describes the success of a combined method, i.e., blended learning
[6]. These concepts are reflected in the design of Raditorium.com. Based on the categories defined by Kerres (content, communication, and construction)
[7] and inspired by the behavioral approaches of Gagnés [8], Kerresʼs 3–2-1 model defines specific parameters that allow didactically useful
structuring [7]. The preparation of learning material (content) is followed by communication and cooperation (communication) and finally by a self-check (construction) [9].
The learning and motivation concepts used in Raditorium.com are cursory and associative learning as well as gamification to increase the intrinsic motivation of students. Cursory reading or learning corresponds to the SQ3R method. By skimming the content to be
learned, the key points can be grasped in a short period of time. This information
is then reviewed and discussed in depth during the in-person class [10]. Gamification meaning game-like presentation of new learning content helps to ensure higher productivity
while simultaneously increasing user satisfaction [11]. Bettina Baesler, co-founder of the online learning platform LernRad, describes
higher long-term learning success as a result of independent learning [12]. Raditorium.com is also based on this principle.
The goal of the present study is to compare the evaluation results of students at
our site before and after the introduction of the online learning platform Raditorium.com in order to quantify the benefits of the new learning platform.
Materials and Methods
Our learning concept is comprised of three important components: 1) preparation, 2)
in-person class, and 3) follow-up. Raditorium.com represents a key medium particularly for preparation and follow-up.
Raditorium.com was established as an online learning platform to accompany classes at the local
campus for students in their clinical semesters. It provides students with free access
to additional learning materials and methods in order to increase the value of in-person
classes and to enhance learning success.
This analysis was approved by the local ethics committee.
The preparation for a particular in-person class consists of a handout for students
and a patient case suitable for the particular class including image material that
can be processed digitally with conventional radiological tools. Both content and construction are communicated as initial learning activities as in a preliminary stage of a flipped classroom
[13]. The cursory reading principle is used. Our teaching experience has shown that students can have
difficulty, particularly in the case of complex subjects (e.g., magnetic resonance
imaging, MRI), depending on their prior knowledge to fully comprehend in-person classes
and thus to achieve the greatest possible learning success. This is expressed, for
example, as a lack of participation as well as difficulty articulating and discussing
specific comprehension questions.
Preparation principle
In the case of cursory learning, i.e., when reading or learning technical or informational texts, the reader
should begin by skimming the text to obtain an initial impression of the content.
When the text is then subsequently closely read, the detailed content supplements
the already acquired preliminary information resulting in higher learning success.
Our teaching concept uses precisely this strategy and allows students to get an initial
impression of learning content based on handouts, lecture slides, and lecture-relevant
case studies. Dosing of preliminary information is extremely important in order to
prevent students from being overwhelmed and to increase motivation to analyze the
content in advance. The presentation of the content also plays an important role since
visually interesting learning materials promise to be more engaging [13]. Thus, e-learning by distributing is ensured by the provision of important materials. The learning process thus takes
place with support but is typically independent as recommended by Bettina Baesler
[14].
Follow-up principle
Follow-up mainly consists of a quiz with questions based on the preceding lecture
to be answered on Raditorium.com. The responses to the questions are evaluated by the next class and are briefly discussed
at the start of the class. As a result, learning mistakes and questions can be addressed
quickly. The brief recapitulation also helps to achieve long-term learning success.
The digital learning content is optimally linked to in-person teaching in terms of
a hybrid teaching concept. The fact that the learning content is still fresh in the
mind at the time of the in-person class results in significant added value. This is
an advantage compared to other learning methods in which the student evaluation and
difficulties understanding technical content are often not known until the end of
the semester. To encourage student motivation, we used a point system for the quiz
and gamification as the motivation principle. The integration of games in the learning process, e.g.,
the collection of points, and the possibility of being named the winner at the end
of the semester can make learning more exciting and motivate students in this established
principle. At the end of the semester, the students with the most points receive an
award. According to Reinmann, e-learning by interacting is combined here with e-learning by distributing. At the same time teachers receive important information about the level of knowledge
and understanding of students and can adapt future courses. As in high-performance
sports, in-person classes are able to be adapted on a more individual basis during
the entire semester in order to achieve the best possible learning success for the
students.
Collecting evaluation results
The concept and the effect of Raditorium.com were analyzed by evaluating the rankings from the studentsʼ semester evaluations
starting in winter semester (2018/19 (WS 18/19), i.e., for a total of seven semesters
prior to the introduction of Raditorium.com and 3 semesters after the introduction. The inclusion of evaluation results starting
in WS 18/19 shows the expected influence of the corona virus pandemic and additionally
allows comparison of the results after the pandemic with those before the pandemic.
The numeric evaluation is first assessed starting with summer semester 2020 (SS 20)
because there has since been a change in the evaluation sheets used by the local medical
school. School grades 1–5 were used with 1 indicating complete agreement and 5 indicating
complete disagreement.
Results
The present results are anonymized and were obtained from the semester evaluations
of the medical school at Christian-Albrechts-University Kiel. The evaluation participants
were medical students in their first clinical year.
The teaching of students was fully expanded to include the online platform Raditorium.com in the Clinic for Radiology and Neuroradiology at the university hospital Schleswig-Holstein,
Campus Kiel in SS 22. In WS 21/22, initial learning content was offered online as
a test phase. A positive trend in the results of student evaluations of the medical
school – compared to the corona virus pandemic – was already able to be identified.
In SS 22, an improvement in ranking to 16 was seen followed by a further improvement
to a ranking of 2 in SS 22 and 23 ([Fig. 1]). In the four evaluations prior to complete implementation of Raditorium.com, the average ranking of radiology among clinical subjects was 25. It subsequently
improved to 1.6 after the introduction. Although the conditions for teachers in clinical
subjects were the same during the corona virus pandemic, the evaluation results in
the three semesters prior to the introduction of Raditorium.com can be partly attributed to the corona virus pandemic. However, the average evaluation
results with Raditorium.com are significantly better than before the pandemic when the average was 13. The best
ranking in the teaching evaluation prior to the introduction of Raditorium.com was 10. This increased to 1 after the introduction of Raditorium.com.
Fig. 1 Ranking of the Clinic for Radiology and Neuroradiology at the university hospital
Schleswig-Holstein, Campus Kiel between winter term 2018/19 and summer term 2023.
Grading of the evaluation questions and the number of questions regarding teaching
content (grade 1–5; 1: strongly agree, 5: disagree) has been expanded at this location
since the corona virus pandemic and this expanded system has been used since then.
A significant improvement in all points ([Table 1]) has been seen since this time. The mean value of all evaluation questions from
WS 20/21 to SS 23 even improved from 3.14 to 1.2. The final grade for the entire course
improved from 3.5 to 1.2. Raditorium.com can be identified as a central change with a causal effect on this improvement.
Table 1 Questions and results of the student evaluation of teaching at the school of medicine
showing a comparison of the mean results of summer semester 2020, winter semester
2020/21, and summer semester 2023 with 1 indicating full agreement and 5 indicating
disagreement.
|
Questions regarding learning content
|
Mean values
|
|
SS 20
|
WS 20/21
|
SS 23
|
|
1
|
All information needed for the course was available in a timely manner.
|
2
|
3.5
|
1.2
|
|
2
|
The structure of the course content was clear.
|
2.1
|
3.7
|
1.3
|
|
3
|
The learning objectives of the course were clearly communicated.
|
2
|
3.7
|
1.2
|
|
4
|
The course supported my interest in the learning content.
|
1.6
|
2.9
|
1.2
|
|
5
|
I learned a lot of new information in the class.
|
1.9
|
2.8
|
1.4
|
|
6
|
The provided materials facilitated good exam preparation.
|
2.1
|
3.5
|
1.1
|
|
7
|
The tempo of the course was appropriate for me.
|
2.9
|
2.3
|
1.1
|
|
8
|
The material to be learned in the course was appropriate for me.
|
2.9
|
2.4
|
1.1
|
|
9
|
Overall, I am very satisfied with the course.
|
2.1
|
3.5
|
1.2
|
In the evaluation of SS 23, 91.3% of students gave the courses a very positive rating.
Evaluation of the open comments shows a definitive relationship to the online learning
platform Raditorium.com. Students are in favor of independent learning and describe the advantage of being
able to subjectively adjust to their own learning tempo.
[Fig. 2] shows a graph of the differences in the average evaluation between WS 20/21 and
SS 23. The most important improvements were seen with regard to the following points:
1) All information needed for the course was available in a timely manner, 2) The
content of the course had a clear structure, 3) The learning objectives of the course
were clearly communicated, 6) The provided materials allowed proper exam preparation,
and 9) I am very satisfied with the course.
Fig. 2 Graphic representation of the mean values from student evaluation of teaching from
winter semester (WS) 2020/21 until summer semester (SS) 2023 based on rating of questions
1–9 from [Table 1].
In-depth study of the learning content is also important and was performed by 86.4%
of the students. The game-like presentation of learning content, e.g., in the form
of quizzes, is described as particularly engaging in the open comments section of
the evaluation. Therefore, 95.2% of students stated that they felt well prepared for
the examination. Particular emphasis was placed on the four main areas of Raditorium.com: 1) Uniform lecture slides, 2) Handouts, 3) Lecture-based case studies, and 4) Self-tests.
Digital access to lecture slides is considered especially helpful for subsequent follow-up.
Students can follow lectures without having to write down all necessary information.
This allows them to concentrate fully on what is being said. In addition, the early
availability of learning content makes it possible for students to prepare topics
before class. Students stated that this allows them to follow content better and to
participate more actively in questions and examples.
The digital handouts for courses provide an overview of examination-relevant content
and thus have a positive effect on individual course and examination preparation.
Students highlight the advantages of flexible access to such summaries in the free-text
comments since the content can be read and learned on their preferred device.
On the whole, the evaluation results thus show an enormous improvement in the satisfaction
of students, particularly due to Raditorium.com being established as a hybrid teaching medium. These results confirm our innovative
teaching approach and are the basic requirement for a modern radiological teaching
concept.
Conclusion
Training regarding imaging modalities is extremely important due to the their key
interdisciplinary role and the strong synergy between radiology and other disciplines.
Finally due to the plurality of imaging and interventional methods, radiology is a
core discipline. Using learning and motivation principles like cursory learning and gamification, complex learning content can be processed as effectively as possible, resulting in
long-term learning success. Raditorium.com combines the needs of students and effective learning strategies with a focus on
the preparation and follow-up of in-person classes. As an online learning platform,
it provides one of foundations of hybrid teaching. This concept can be used for additional
areas of didactics in medicine in order to communicate learning content to students
in a practical manner while ensuring long-term success.
Discussion
In the wake of digitalization, a hybrid learning concept is highly desirable for students
at medical schools. Digital learning platforms as exemplified by Raditorium.com allow flexible access to learning content for preparation and follow-up. Only a digital
device is needed. Game-like websites use the concept of gamification, for example, in the form of integrated self-tests with rankings, to ensure greater
intrinsic motivation and thus higher learning success. The provision of relevant content
also allows better focus and participation on the part of students during in-person
teaching since the students do not need to take notes and can actively participate
during class time. The teaching evaluations in particular show that the majority of
surveyed students felt well prepared for in-person classes due to the supportive function
of preparation and follow-up and this increased their motivation for further study
of the topics.
The online learning platform Raditorium.com is the core of our concept and offers students simple access to all relevant materials
so that they can implement the principle of preparation and follow-up. Studies performed
to date show the positive effect of e-learning sites on learning success, i.e., a
study by Scott et al. from the year 2017. In this study, 76% of surveyed medical students
stated that they use e-learning resources of other universities or institutions [15]. In our opinion this need should ideally be met by each medical school individually
since the content can thus be optimally adjusted to the schoolʼs own courses, content,
examinations, and teaching concepts.
The development of Raditorium.com required prior analysis of e-learning in order to be able to integrate the online
learning platform into the structure of the school in a didactically useful manner.
It is important to note that e-learning is significantly more significant than the
simple provision of texts and multimedia materials on platforms on the Internet [16]. The results of Raditorium.com provide new insight into the proper use of a digital learning platform based on the
concept: content, communication, and construction. However, it is important to determine the best presentation of these areas prior
to designing such an online learning platform. The mere availability of a digital
learning platform does not ensure a didactically useful teaching concept. Finally,
as a result of the increasing number of digital online teaching options, a number
of digital learning resources in medicine and radiology are already available and
could provide initial orientation to facilitate the creation of an own website [17].
When developing Raditorium.com, the preparation of the learning content as well as the creation of the website were
very time-intensive. The initial creation of a digital learning platform requires
a wide range of competencies in the field of IT and digitalization [18] and can be very expensive. However, as the medical schoolʼs own solution, Raditorium.com allows a high degree of flexibility in the preparation of learning content. New content
can be entered easily and quickly in a well-structured website. This results in increased
learning success for students as well as a time savings for instructors. The linking
of external learning content (third-party providers) is advantageous for several reasons
but it may not be coordinated with oneʼs own teaching concept, e.g., with regard to
unnecessary redundancy and content that is not suitable for the particular semester.
A further advantage of Raditorium.com is the just-in-time evaluation of self-tests completed by students. This allows early
detection of learning deficits among students that can be immediately corrected with
additional information provided via Raditorium.com or in the next in-person class. Upcoming classes can also be adapted.
One main aspect of an online learning platform is data protection. It is important
to allow students secure access to learning content even online. Therefore, Raditorium.com is password-protected and meets the requirements for a secure website – by anonymizing
data among other things [19].
It should be noted that the present evaluation data only show the development for
one year. Since the online learning platform Raditorium.com was first fully implemented in SS 22, long-term results regarding academic performance
or long-term improvement in student satisfaction cannot yet be shown. However, the
results obtained to date show a causal relationship between the evaluation results
and Raditorium.com. The online learning platform became fully established between evaluation points
SS 22 and SS 23 as a key medium that combines flexibility and individuality and meets
the requirements for hybrid teaching. Since the introduction of Raditorium.com was the main project in this period, other third-party variables can be largely ruled
out. However, to be able to compare the relationship between Raditorium.com and the learning success of individual students, additional long-term studies are
needed in the future. Nevertheless, the results clearly show the benefit of digital
learning platforms.
