Human Factors in the Large: Experiences from Denmark, Finland and Canada in Moving Towards Regional and National Evaluations of Health Information System Usability

Summary

Objectives: The objective of this paper is to explore approaches to understanding the usability of health information systems at regional and national levels.

Methods: Several different methods are discussed in case studies from Denmark, Finland and Canada. They range from small scale qualitative studies involving usability testing of systems to larger scale national level questionnaire studies aimed at assessing the use and usability of health information systems by entire groups of health professionals.

Results: It was found that regional and national usability studies can complement smaller scale usability studies, and that they are needed in order to understand larger trends regarding system usability. Despite adoption of EHRs, many health professionals rate the usability of the systems as low. A range of usability issues have been noted when data is collected on a large scale through use of widely distributed questionnaires and websites designed to monitor user perceptions of usability.

Conclusion: As health information systems are deployed on a widespread basis, studies that examine systems used regionally or nationally are required. In addition, collection of large scale data on the usability of specific IT products is needed in order to complement smaller scale studies of specific systems.

Keywords

Health informatics - human factors - usability - health information systems - evaluation

• References

• 1 Nielsen J. Usability engineering.. San Diego: Academic Press, Inc.; 1993
• 2 Shackel B, Richardson S. editors. Human factors for informatics usability.. New York: Cambridge University Press; 1991
• 3 Kushniruk AW, Patel V. Cognitive and usability engineering approaches to the evaluation of clinical information systems.. J Biomed Inform 2004; 37 (01) 56-76.
• 4 Beuscart-Zéphir MC, Pelayo S, Ancequx F, Meaux J, Degroisse M, Degoulet P. Impact of CPOE on doctor–nurse cooperation for the medication ordering and administration process.. Int J Med Inform 2005; 74 (07) 629-41.
• 5 Kaufman DR, Patel VL, Hiliman PC, Morin J, Pevzner RS, Weinstock RG. et al. Usability in the real world: assessing medical information technologies in patients’ homes.. J Biomed Inform 2003; 36 (01) 45-60.
• 6 Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices.. J Biomed Inform 2003; 36 (01) 23-30.
• 7 Kjeldskov J, Skov M, Stage J. Instant data analysis: Conducting usability evaluations in a day.. NordiCHI ’04, October 23-27, 2004. Tampere,; Finland;: 2004
• 8 Kushniruk AW, Triola M, Borycki EM, Stein B, Kannry J. Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application.. Int J Med Inform 2005; 74 7-8 519-26.
• 9 Winblad I, Reponen J, Hämäläinen P, Kangas M. Informaatio- ja kommunikaatioteknologian käyttö Suomen terveydenhuollossa vuonna 2007.. Tilanne ja kehityksen suunta. Stakesin raportteja 37/2008. [Health Care ICT and eHealth in Finland 2007. Status report and future trends. Reports 37/2008] Helsinki. 2008 http://www.stakes.fi/verkkojulkaisut/raportit/R37-2008-VERKKO.pdf. In Finnish.:
• 10 Lääveri T, Virtanen A, Paajanen H, Ahtola H, Konki K. Potilaskertomusjärjestelmien käytettävyysselvitys osoitti: Lääkärien mielipiteitä ei ole kuunneltu.. [Study on Usability of Electronic Health Record Systems Shows: Physicians’ Opinions Have Been Ignored].. Finnish Medical Journal 6, 2008. additional part on Electronic health record systems,; 3-33. In Finnish.:
• 11 Viitanen J, Hyppönen H, Lääveri T, Vänskä J, Reponen J, Winblad I. National questionnaire study on clinical ICT systems proofs: Physicians suffer from poor usability.. Int J Med Inform 2011; 80 (10) 708-25.
• 12 Hyppönen H, Faxvaag A, Gilstad H, Hardardottir G, Jerlvall L, Kangas M. et al. Nordic eHealth indicators.. Organization of research, first results and the plan for the future.; TemaNord: 2013. 522, http://dx.doi.org/10.6027/TN2013-522, Accessed 5.2.2014
• 13 Viitanen J, Nieminen M, Hyppönen H, Lääveri T. Finnish physicians’ experiences with computer-supported patient information exchange and communication in clinical work.. Int J Electron Healthc 2011; 6 2-4 153-73.
• 14 Martikainen S, Viitanen J, Korpela M, Lääveri T. Physicians’ experiences of participation in healthcare IT development: Willing but not able.. Int J Med Inform 2012; 81 (02) 98-113.
• 15 Act on the electronic processing of client data in social and health care services.. Available (in Finnish and Swedish) from http://plus.edilex.fi/kela/fi/lainsaadanto/20070159. Accessed 5.2. 2014
• 16 Hyppönen H, Doupi P, Hämäläinen P, Komulainen J, Nykänen P, Suomi R. Towards a national health information system evaluation.. Stud Health Technol Inform 2010; 160 (02) 1216-20.
• 17 Hyppönen H, Faxvaag A, Gilstad H, Hardardottir GA, Jerlvall L, Kangas M. et al. Nordic eHealth indicators: organization of research, first results and plan for the future.. Stud Health Technol Inform 2013; 192: 273-7.
• 18 National Physician Survey,. 2010 Retrieved from: http:www.nationalphysiciansurvey.ca
• 19 Kushniruk A, Owston R, Ho F, Pitts K, Wideman H, Brown C. et al. Design of the VULab: A quantitative and qualitative tool for analyzing use of on-line health information resources.. Proceedings of ITCH 2007 February 2007.
• 20 Kushniruk AW, Patel C, Patel VL, Cimino JJ. “Tel-evaluation” of information systems: An integrative approach to design and evaluation of Web-based systems.. Int J Med Inform 2001; 61 (01) 45-70.
• 21 Felciano RM, Altman RB. Lamprey: Tracking users on the World Wide Web.. In: Proceedings of the 1996 AMIA Ann Fall Symposium;. 1996 p. 757-61.
• 22 Kushniruk A, Beuscart-Zéphir MC, Grzes A, Borycki EM, Watbled L, Kannry J. Increasing the safety of healthcare information systems through improved procurement: Toward a framework for selection of safe healthcare systems.. Healthc Q 2010; 13: 53-8.
• 23 Carvalho CJ, Borycki EM, Kushniruk A. Ensuring the safety of health information systems: Using heuristics for patient safety.. Healthc Q 2009; 12: 49-54.
• 24 Schumacher RM, Webb JM, Johnson KR. How to select an electronic health record system that healthcare professionals can use.. User centric, Inc.;; 2009. Available from: http://www.user-centric.com/sites/usercentric.com/files/usercentric-ehr-white-paper.pdf
• 25 Jokela T, Laine J, Nieminen M. Usability in RFP’s: The current practice and outline for the future.. Lecture notes in computer science; 2013; vol. 8005 p. 101-6. Available online from: http://link.springer.com/chapter/10.1007%2F978-3-642-39262-7_12.
• 26 ISO 9241-11/ISO 9241. Ergonomic requirements for office work with visual display terminals, Part 11: Guidance on Usability.. Geneve:: International Organization for Standardization, Geneve;; 1998
• 27 ISO 9241-210 International Standard:. Ergonomics of human-system interaction – Part 210: Human-centred Design for Interactive Systems.. First edition 2010-03-15, 2010 Reference number ISO 9241-210:2010(E).
• 28 Hertzum M. Images of usability.. Int J Human-Comp Interact 2010; 26: 567-600.
• 29 Davis LS. Problems facing large health information systems.. Proceedings of the Annual ACM Conference, 1973, Atlanta, USA. New York, NY, USA: ACM; 1973
• 30 DeLone WH, McLean ER. TheDelone and McLean model of information systems success: A ten-year update.. Journal of Management Information Systems 2003; 19: 9-30.
• 31 Cooper A. Inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity.. Sams Publishing;; 1998
• 32 Fern X, Komireddy C, Grigoreanu V, Burnett M. Mining problem-solving strategies from HCI data.. Trans Comput Hum Interact; 2010. 17(1).
• 33 Microsoft Health.. Common User Interface.. http://www.mscui.net Accessed May 4 2014