Download PDF
Yearb Med Inform 2013; 22(01): 34-46
DOI: 10.1055/s-0038-1638830
Original Article
Georg Thieme Verlag KG Stuttgart

Evidence Based Health Informatics: 10 Years of Efforts to Promote the Principle

Joint Contribution of IMIA WG EVAL and EFMI WG EVAL
M. Rigby
1   Keele University, School of Public Policy and Professional Practice, Keele, United Kingdom
,
E. Ammenwerth
2   UMIT, University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria
,
M.-C. Beuscart-Zephir
3   INSERM-CIC-IT, CHU Lille, Lille, France
,
J. Brender
4   Aalborg University, Dept. of Health Science and Technology, and Virtual Center for Health Informatics, Aalborg, Denmark
,
H. Hyppönen
5   National Institute for Health and Welfare, Information Department, Helsinki, Finland
,
S. Melia
6   Telefonica UK Ltd., Slough, United Kingdom
,
P. Nykänen
7   Univ. of Tampere, School of Information Sciences, Centre for Information and Systems, eHealth, Tampere, Finland
,
J. Talmon
8   Maastricht University, School for Public Health and Primary Care: Caphri, Maastricht, The Netherlands
,
N. de Keizer
9   Academic Medical Center, Department of Medical Informatics, Amsterdam, The Netherlands
› Author Affiliations
Further Information

Publication History

Publication Date:
05 March 2018 (online)

Also available at
   Permissions and Reprints

Summary

Objectives: To present the importance of Evidence-based Health Informatics (EBHI) and the ethical imperative of this approach; to highlight the work of the IMIA Working Group on Technology Assessment and Quality Improvement and the EFMI Working Group on Assessment of Health Information Systems; and to introduce the further important evaluation and evidence aspects being addressed.

Methods: Reviews of IMIA, EFMA and other initiatives, together with literature reviews on evaluation methods and on published systematic reviews.

Results: Presentation of the rationale for the health informatics domain to adopt a scientific approach by assessing impact, avoiding harm, and empirically demonstrating benefit and best use; reporting of the origins and rationale of the IMIA- and EQUATOR-endorsed Statement on Reporting of Evaluation Studies in Health Informatics (STARE-HI) and of the IMIA WG's Guideline for Good Evaluation Practice in Health Informatics (GEP-HI); presentation of other initiatives for objective evaluation; and outlining of further work in hand on usability and indicators; together with the case for development of relevant evaluation methods in newer applications such as telemedicine. The focus is on scientific evaluation as a reliable source of evidence, and on structured presentation of results to enable easy retrieval of evidence.

Conclusions: EBHI is feasible, necessary for efficiency and safety, and ethically essential. Given the significant impact of health informatics on health systems, care delivery and personal health, it is vital that cultures change to insist on evidence-based policies and investment, and that emergent global moves for this are supported.

Keywords

Evidence-based health informatics - evaluation - evidence - health informatics - ethics
 

  • References

  • 1 http://www.imia-medinfo.org/new2/GA/2012beijing/1210GA-beijing-president-WHO.pdf (accessed 31 January 2013).
    PubMed
  • 2 Rigby M, Ammenwerth E, Talmon J, Nykanen P, Brender J, de Keizer N. Health Informatics 3.0 and other Increasingly Dispersed Technologies Require Even Greater Trust: Promoting Safe Evidence-based Health Informatics. In: Geissbuhler A, Kulikowski CA. editors. IMIA Yearbook of Medical Informatics 2011: Towards Health Informatics 3.0. Stuttgart: Schattauer; 2011. p. 105-11.
    Google Scholar
  • 3 Beauchamp T, Childress JF. Principles of Biomedical Ethics. 6th ed. New York: Oxford University Press; 2009
    Google Scholar
  • 4 European Commission. Communication from the Commission on the Precautionary Principle, COM(2000) 1. Brussels: 2000
    Google Scholar
  • 5 Fisher E, Jones J, von Schomberg R. editors. Implementing the Precautionary Principle: Perspectives and Prospects. Cheltenham, UK and Northampton. MA, US: Edward Elgar; 2006
  • 6 Ammenwerth E, Shaw NT. Bad health informatics can kill — is evaluation the answer?. Methods Inf Med 2005; 44 (1) 1-3.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 7 Web list “Bad Health Informatics can Kill”. (accessible via the Bad Health Informatics link at http:// iig.umit.at/efmi) (accessed 31 January 2013).
    PubMedGoogle Scholar
  • 8 http://business-ethics.com/ (accessed 31 January 2013).
    PubMed
  • 9 http://www.ibe.org.uk/ (accessed 31 January 2011).
    PubMed
  • 10 Patrick JD. The Validity of Personal Experiences in Evaluating HIT. Appl Clin Inf 2010; 1: 462-5.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 11 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=C0NSLEG:1993L0042:20071011:en:PDF (accessed 31 January 2013).
    PubMed
  • 12 Haynes RB, Hayward RS, Jadad AR, Sebaldt RJ. Evidence based health informatics: an overview of the Health Information Research Unit at Mc-Master University. Leadersh Health Serv 1996; May-Jun 5 (3) 41-4.
    PubMedGoogle Scholar
  • 13 Rigby M. Evaluation: 16 powerful reasons why not to do it-and 6 over-riding imperatives. In: Patel V, Rogers R, Haux R. editors. Proceedings of the 10th World Congress on Medical Informatics (Medinfo 2001). Amsterdam: IOS Press; 2001. p. 1198-202.
    Google Scholar
  • 14 Ammenwerth E, Brender J, Nykänen P, Prokosch H-U, Rigby M, Talmon J. et al. Visions and strategies to improve evaluation of health information systems. Reflections and lessons based on the HIS-EVAL workshop in Innsbruck. Int J Med Inf 2004; Jun 30 73 (6) 479-91.
    CrossrefPubMedGoogle Scholar
  • 15 Ammenwerth E. Health technology assessment. Findings from the Section on Assessing Information Technologies for Health. Yearb Med Inform 2006; 16-9.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 16 Ammenwerth E, Schnell-Inderst P, Siebert U. Vision and Challenges of Evidence-Based Health Informatics: A case study of a CPOE meta-analysis. Int J Med Inform 2010; 79: e83-e88.
    CrossrefPubMedGoogle Scholar
  • 17 Sackett D, Rosenberg W, Gray J, Haynes R, Richardson S. Evidence based medicine: what it is and what it isn't. BMJ 1996; 312 (7023) 71-2.
    CrossrefPubMedGoogle Scholar
  • 18 Ammenwerth E. Evidence based health informatics. Stud Health Technol Inform 2010; 151: 427-34.
    PubMedGoogle Scholar
  • 19 Wyatt J. Assessing and improving evidence based health informatics research. Stud Health Technol Inform 2010; 151: 435-45.
    PubMedGoogle Scholar
  • 20 Mantas J, Ammenwerth E, Demiris G, Hasman A, Haux R, Hersh W. et al. Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics. IMIA White Paper. Methods Inf Med 2010; 49 (2) 1-16.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 21 Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI — Statement on reporting of evaluation studies in Health Informatics. Int J Med Inform 2009; 78 (1) 1-9.
    CrossrefPubMedGoogle Scholar
  • 22 http://www.equator-network.org/resource-centre/library-of-health-research-reporting/reporting-guidelines/other-reporting-guidelines/ (accessed 31 January 2013).
    PubMed
  • 23 Nykänen P, Brender J, Talmon J, de Keizer N, Rigby M, Beuscart-Zephir M-C. et al. Guideline for good evaluation practice in health informatics (GEP-HI). Int J Med Inform 2011; 80: 815-27.
    CrossrefPubMedGoogle Scholar
  • 24 de Keizer NF, Talmon J, Ammenwerth E, Brender J, Nykanen P, Rigby M. Mini Stare-HI: Guidelines for Reporting Health Informatics Evaluations in Conference Papers. In: Safran C, Reti S, Marin HF. editors. MEDINFO 2010 — Proceedings of the 13th World Congress on Medical Informatics. Amsterdam: IOS Press; 2010. p. 1206-10.
    Google Scholar
  • 25 Ammenwerth E, de Keizer N. An inventory of evaluation studies of information technology in health care: Trends in evaluation research 1982 — 2002. Methods Inf Med 2005; 44: 44-56.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 26 Ammenwerth E, de Keizer N. Evaluation Database — A web-based inventory of evaluation studies in medical informatics. Institute for Health Information Systems, University for Health Sciences, Medical Informatics and Technology; Austria: (on-going) (available at http://evaldb.umit.at) (accessed 31 January 2013).
    Google Scholar
  • 27 Rigby M, Brender J, Beuscart-Zephir M-C, Hypponen H, Nykänen P, Talmon J. et al. Next Steps in Evaluation and Evidence — from Generic to Context-Related. In: Moen A, Andersen SK, Aarts J, Hurlen P. editors. User Centred Networked Health Care — Proceedings of MIE 2011. IOS Press; 2011. p. 208-12.
    Google Scholar
  • 28 Melia S. The challenge of evaluating telehealth interventions — Dissertation in part fulfilment of the award of MBA (Health Executive). Keele University; 2012
    Google Scholar
  • 29 Nykänen P, Kaipio J, Kuusisto A. Evaluation of the Finnish nursing classification system and four nursing documentation systems in Finland — lessons learned and directions or the future. Int J Med Inform 2012; 81 (8) 505-20.
    PubMedGoogle Scholar
  • 30 Kopanitsa G, Tsvetkova Z, Veseli H. Analysis of metrics for usability evaluation of electronic health record systems. In: Blobel B. editor Large-scale projects in eHealth. Amsterdam: IOS Press; 2012. p. 129-33.
    Google Scholar
  • 31 Anttila M. Adoption of information technology based patient education in psychiatric nursing. PhD dissertation. University of Turku; 2012 Accessible at: http://www.doria.fi/handle/10024/74662 (accessed 31 January 2013).
    PubMedGoogle Scholar
  • 32 Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C. et al. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review. Int J Med Inform 2012; 81 (10) e83-92.
    CrossrefPubMedGoogle Scholar
  • 33 Scott PJ, Briggs J. STAT-HI: A socio-technical assessment tool for health informatics implementations. Open Med Inform J 2010; 4: 214-20.
    CrossrefPubMedGoogle Scholar
  • 34 Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001; Apr 14 357 (9263) 1191-4.
    CrossrefPubMedGoogle Scholar
  • 35 Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM. et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. BMJ 2003; 326 (7379) 41-4.
    CrossrefPubMedGoogle Scholar
  • 36 Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999; Nov 27 354 (9193) 1896-900.
    CrossrefPubMedGoogle Scholar
  • 37 Talmon J, Ammenwerth E, Geven T. The quality of reporting of health informatics evaluation studies: a pilot study. Stud Health Technol Inform 2007; 129 (Pt 1) 193-7.
    PubMedGoogle Scholar
  • 38 de Keizer NF, Talmon J, Ammenwerth E, Brender J, Rigby M, Nykanen P. Systematic Prioritization of the STARE-HI Reporting Items. An Application to Short Conference Papers on Health Informatics Evaluation. Methods Inf Med. 2011 Mar 4 50. (3).
    PubMedGoogle Scholar
  • 39 Brender J. Handbook of Evaluation Methods for Health Informatics. New York: Academic Press; 2006. p. 361.
    Google Scholar
  • 40 Friedman CP, Wyatt JC. Evaluation Methods in Biomedical Informatics. New York: Springer; 2006
    Google Scholar
  • 41 Friedman CP, Altman RB, Kohane IS, McCormick KA, Miller PL, Ozbolt JG. et al. Training of the next generation of informaticians: the impact of “BISTI” and bioinformatics — a report of the American College of Medical Informatics. J Am Med Inform Assoc 2004; 11 (3) 167-72.
    CrossrefPubMedGoogle Scholar
  • 42 Gardner RM, Overhage JM, Steen EB, Munger BS, Holmes JH, Williamson JJ. et al. Core Content for the Subspecialty of Clinical Informatics. J Am Med Inform Assoc 2009; 16: 153-7.
    CrossrefPubMedGoogle Scholar
  • 43 Borycki E, Kushniruk A. Identifying and preventing technology-induced error using simulations: application of usability engineering techniques. Healthc Q 2005; 8 Spec No 99-105.
    CrossrefPubMedGoogle Scholar
  • 44 Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application. Int J Med Inform 2005; Aug 74 (7- 8) 519-26.
    CrossrefPubMedGoogle Scholar
  • 45 International Electrotechnical Commission. Medical Electrical Equipment — General requirements for basic safety and essential performance. Collateral Standard. Usability. Geneva: International Electrotechnical Commission; 2007. Report No.: IEC 60601-1-6).
    Google Scholar
  • 46 International Electrotechnical Commission. Medical Electrical Equipment — General requirements for basic safety and essential performance. Collateral Standard. Usability. Geneva: International Electrotechnical Commission; 2010. Report No.: IEC 60601-1-6.
    Google Scholar
  • 47 Beuscart-Zephir MC, Elkin P, Pelayo S, Beuscart R. The human factors engineering approach to biomedical informatics projects: state of the art, results, benefits and challenges. Yearb Med Inform 2007; 109-27.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 48 Beuscart-Zephir MC, Pelayo S, Borycki E, Kushniruk A. Human Factors Considerations in Health IT Design and Development. In: Carayon P. editor. Human Factors and Ergonomics in Health Care and Patient Safety 2nd Edition. CRC Press; 2012. p. 649-70.
    Google Scholar
  • 49 Borycki E, Kushniruk A. Where do technology-induced errors come from? Towards a model for conceptuallizing and diagnosing errors caused by technology. In: Kushniruk A, Borycki E. editors. Human, Social and Organizational Aspects of Health Information Systems. Hershey, NY: Idea Group; 2008
    Google Scholar
  • 50 Beuscart-Zephir MC, Borycki E, Carayon P, Jaspers MW, Pelayo S. Evolution of Human factors Research and Studies of Health information technologies: the Role of Patient safety. Yearb Med Inform. 2013 Seitenzahlen.
    PubMedGoogle Scholar
  • 51 Magrabi F, Aarts J, Nohr C, Baker M, Harrison S, Pelayo S. et al. A comparative review of patient safety initiatives for national health information technology. Int J Med Inform 2013; May 82 (5) e139-48.
    CrossrefPubMedGoogle Scholar
  • 52 Marcilly R, Beuscart-Zephir MC, Pelayo S. Seeking Evidence to support Usability Principles for medication-related CDS functions. 2013
    PubMedGoogle Scholar
  • 53 Peute LW, Spithoven R, Bakker PJ, Jaspers MW. Usability studies on interactive health information systems; where do we stand?. Stud Health Technol Inform 2008; 136: 327-32.
    PubMedGoogle Scholar
  • 54 Lawrence M, Olesen F. Indicators of Quality in Health Care. Eur J Gen Pract 1997; 3 (3) 103-8.
    CrossrefPubMedGoogle Scholar
  • 55 Hyppönen H, Ammenwerth E, Faxwaag A, Nohr C, Walldius A. eHealth indicators: results of the MIE 2011 workshop. Stud Health Technol Inform 2012; 180: 328-32.
    PubMedGoogle Scholar
  • 56 Hyppönen H, Vuokko R, Mäkelä-Bengs P, Doupi P, Saranto K, Mäkelä M. Impacts of structuring the patient record: a systematic review protocol and results of previous reviews. [Submitted to Int J Med Inform, under review].
    PubMedGoogle Scholar
  • 57 DeLone WH, McLean ER. The DeLone and McLean Model of Information Systems Success: a ten-year update. Journal of Management Information Systems 2003; 19 (4) 9-30.
    CrossrefPubMedGoogle Scholar
  • 58 Lampe K, Mäkelä M. editors. HTA Core Model for medical and surgical interventions. First Public Draft (Revised) EUnetHTA: Work Package 4, 2007, available at http://www.eunethta.eu/outputs/hta-core-model-medical-and-surgical-interventions-10r (accessed 31 January 2013).
    PubMedGoogle Scholar
  • 59 Hyppönen H, Ammenwerth E, de Keizer N. Exploring a methodology for eHealth indicator development. Stud Health Technol Inform 2012; 180: 338-42.
    PubMedGoogle Scholar
  • 60 Hyppönen H, Faxvaag A, Nohr C, Vimarlund V, Hardardottir GA, Reponen J. et al. Nordic eHealth Indicators — Organisation of research, first results and plan for the future. [in press, Nordic Council of Ministers, Tema Nord-series number TN2013:522].
    PubMedGoogle Scholar
  • 61 Rigby M. Evaluation — the Cinderella Science of ICT in Health. In Kulikowski C, Haux R. editors. IMIA Yearbook of Medical Informatics 2006. Stuttgart: Schattauer; 2006. p. 114-20.
    Google Scholar
  • 62 Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008; Jun 77 (6) 386-98.
    CrossrefPubMedGoogle Scholar
  • 63 Cusack CM, Byrne C, Hook JM, McGowan J, Poon EG, Zafar A. Health information technology evaluation toolkit: 2009 Update (Prepared for the AHRQ National Resource Center for Health Information Technology under Contract No. 29004-0016.) AHRQ publication No. 09-0083-EF. Rockville, MD: Agency for Healthcare Research and Quality; June 2009.
    Google Scholar
  • 64 Kidholm K, Bowes A, Dyrehauge S, Granstrom Ekeland A, Flottorp SA, Kvistgaard Jensen L. et al. The MAST Manual. MAST-Model for assessment of tele medicine. http://www.epractice.eu/en/library/315610 (accessed 31 January 2013).
    PubMedGoogle Scholar
  • 65 Lorenzi N, Riley R, Blyth A, Southon G, Dixon B. Antecedents of the people and organizational aspects of medical informatics. J Am Med Inform Assoc 1997; 4 (2) 79.
    CrossrefPubMedGoogle Scholar
  • 66 Berg M. Medical work and the computer-based patient record: a sociological perspective. Methods Inf Med 1998; 7 (3) 294-301.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 67 Goorman E, Berg M. Modelling nursing activities: electronic patient records and their discontents. Nurs Inq 2000; 7 (1) 3-9.
    CrossrefPubMedGoogle Scholar
  • 68 Aarts J, Doorewaard H, Berg M. Understanding implementation: the case of a computerized physician order entry system in a large Dutch university medical center. J Am Med Inform Assoc 2004; 11 (3) 207-16.
    CrossrefPubMedGoogle Scholar
  • 69 Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007; 76 (Suppl 1) 21-7.
    CrossrefPubMedGoogle Scholar
  • 70 Niazkhani Z, Pirnejad H, Aarts J, Adams S, Bal R. Reporting qualitative research in health informatics: REQ-Hl recommendations. Stud Health Technol Inform 2011; 169: 877-81.
    PubMedGoogle Scholar
  • 71 Eysenbach G. CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res. 2011 Dec 31 13. (4) e126. doi: 10.2196/jmir.1923.
    PubMedGoogle Scholar
  • 72 Khoja S, Durrani H, Scott RE, Sajwani A, Piryani U. Conceptual Framework for Development of Comprehensive e-Health Evaluation Tool. Telemed J E Health 2013; 19 (1) 48-53.
    CrossrefPubMedGoogle Scholar
  • 73 Call to Action on Global eHealth Evaluation — Consensus Statement of the WHO Global eHealth Evaluation Meeting. Bellgio: September 2011. available from http://www.healthunbound.org/content/call-action-global-ehealth-evaluation (accessed 31 January 2013).
    Google Scholar
  • 74 Ham C, Hunter DJ, Robinson R. Evidence-based Policy. BMJ 1995; 310 (6972) 71-2.
    CrossrefPubMedGoogle Scholar
  • 75 Walshe K, Rundall TG. Evidence-based Management: From Theory to Practice in Health Care. The Milbank Quarterly 2001; 79: 429-57.
    CrossrefPubMedGoogle Scholar
  • 76 Gray JAM. Evidence-based Healthcare: How to make Health Policy and Management Decisions. 2nd ed. Edinburgh: Churchill Livingstone; 2011. p. 444.
    Google Scholar
  • 77 Rigby M, Ronchi E, Graham S. Evidence for building a smarter health and wellness future — Key messages and collected visions from a Joint OECD and NSF workshop. Int J Med Inform. 2012 Oct 29. pii: S1386-5056(12)00195-5. doi: 10.1016/j. ijmedinf.2012.10.003. [Epub ahead of print].
    PubMedGoogle Scholar