The New Features of the ExaMe Evaluation System and Reliability of Its Fixed Tests

Summary

Objectives: The ExaMe system for the evaluation of targeted knowledge has been in development since 1998. The new features of the ExaMe system are introduced in this paper. Especially, the new three-layer architecture is described.

Besides the system itself, the properties of fixed testsin the ExaMe system are studied. In special detail, the reliabilityof the fixed tests is discussed. The theory background is explained and some limitations of the reliability are pointed out.

Methods for Estimation of the Reliability: Three characteristics used for estimation of reliability of educational tests are discussed: Cronbach’s alpha, standardized item alphaand split half coefficient. The relation between these characteristics and reliability and between characteristics themselves is investigated. In more detail, the properties of Cronbach’s alpha, the characteristics mostly used for the estimation of reliability, are discussed. A confidence interval is introduced for the characteristics.

Results and Conclusions: Since 2000, the serviceability of the ExaMe evaluation system as the supporting evaluation tool has been repeatedly shown at the courses of Ph.D. studies in biomedical informatics at Charles University in Prague. The ExaMe system also opens new possibilities for self-evaluation and distance learning, especially when connected with electronic books on the Internet.

The estimation of reliability of tests contains some limitations. Keeping them in mind, we can still get some information about the quality of certain educational tests. Therefore, the estimation of reliability of the fixed tests is implemented in the ExaMe system.

Keywords

Education - evaluation - Internet - reliability - bioinformatics

• References

• 1 Zvárová J. Education in Methodology for Health Care – EuroMISE. Methods Inf Med 1994; 3.
• 2 Zvárová J, Engelbrecht R, van Bemmel JH. Education in medical informatics, statistics and epidemiology. International Journal for Medical Informatics 1997; 45 (01) (02) 3-8.
• 3 Van Bemmel JH, Zvárová J. Knowledge, Information and Medical Education. Amsterdam: North Hollands Publ Comp 1991
• 4 Haux R, Leven FJ, Möhr J, Protti DJ. editors Special Issue on Health and Medical Informatics Education. Methods Inf Med 1994; 3
• 5 Hasman A. Recommendation for Medical Informatics Training in The Netherlands. Methods Inf Med 1994; 3
• 6 Haux R, Leven FJ. Twenty Years Medical Informatics Education at Heidelberg/Heibronn: Evolution of Specialized Curriculum for Medical Informatics. Methods Inf Med 1994; 33 (03) 285-9.
• 7 Hasman A, Albert A, Wainwright P, Klar R, Sosa M. editors Education and Training in Health Informatics in Europe. IOS Press 1995
• 8 Haux R, Swinkels W, Ball M, Kanup P, Lun KC. editors Special Issue on Health and Medical Informatics Education. Transformation of Healthcare through Innovative Use of Information Technology. Int J Med Inf 1997; 44
• 9 Zvárová J, Zvára K. Evaluation of Knowledge using ExaMe program on the Internet. In Iakovidis I, Maglavera S, Trakatellis A. editors User Acceptance of Health Telematics Applications. Amsterdam: IOS Press; 2000: 145-51.
• 10 Haux R. et al Recommendations of the International Medical Informatics Association (IMIA) on Education in Health and Medical Informatics. Methods Inf Med 2000; 39: 267-77.
• 11 Preckova P, Straka L. Doporuceni mezinarodni asociace pro medicinskou informatiku (IMIA) pro vzdelavani v lekarske a zdravotnicke informatice. Lekar a technika 2001. (Translation of the IMIA recommendations [9])
• 12 Zvárová J, Sva ã ina S. New Czech Postgraduate Doctoral Program in Biomedical Informatics. In Surjan G, Engelbrecht R, Nair P. editors Health Data in the Information Society. Amsterdam: IOS Press; 2002. 90 766-9.
• 13 Sireci SG, Wainer H, Braun H. Psychometrics, Overview. In Encyclopaedia of Biostatistics. New York: Wiley; 1998: 3577-3602.
• 14 Rexová P. Spolehlivost mereni. [Reliability of measurements. In Czech.] Diploma thesis. Department of Probability and Mathematical Statistics, Charles University, Prague 2003
• 15 Novick MR, Lewis C. Coefficient alpha and the internal structure of tests. Technical report, Education Testing Service. Princeton, New Jersey 1966
• 16 The R Project for Statistical Computing Available from http://www.r-project.org/
• 17 Suen HK. Principles of Test Theories. LEA, New Jersey 1990; 71-129.
• 18 Rexová P. Item Analysis of Educational Tests in System ExaME. In Hakl F. editor Doktorandský den ’04, Institute of Computer Science AS CR, MatfyzPress, Prague. 2004