Methods Inf Med 2008; 47(03): 251-259
DOI: 10.3414/ME0512
Original Article
Schattauer GmbH

Prioritizing the Risk Factors Influencing the Success of Clinical Information System Projects

A Delphi Study in Canada
G. Paré
1   Canada Research Chair in Information Technology in Health Care, HEC Montréal, Montreal, Quebec, Canada
,
C. Sicotte
2   University of Montréal, Montreal, Quebec, Canada
,
M. Jaana
3   University of Ottawa, Ottawa, Ontario, Canada
,
D. Girouard
1   Canada Research Chair in Information Technology in Health Care, HEC Montréal, Montreal, Quebec, Canada
› Author Affiliations
Further Information

Publication History

Received: 25 October 2007

accepted: 30 January 2007

Publication Date:
18 January 2018 (online)

Summary

Objective: The aim of this study is to gain a better understanding of the risk factors influencing the success of clinical information system projects.

Methods: This study addresses this issue by first reviewing the extant literature on information technology project risks, and second conducting a Delphi survey among 21 experts highly involved in clinical information system projects in Québec, Canada, a region where government have invested heavily in health information technologies in recent years.

Results: Twenty-three risk factors were identified. The absence of a project champion was the factor that experts felt most deserves their attention. Lack of commitment from upper management was ranked second. Our panel of experts also confirmed the importance of a variable that has been extensively studied in information systems, namely, perceived usefulness that ranked third. Respondents ranked project ambiguity fourth. The fifth-ranked risk was associated with poor alignment between the clinical information systems’ characteristics and the organization of clinical work. The large majority of risk factors associated with the technology itself were considered less important. This finding supports the idea that technology-associated factors rarely figure among the main reasons for a project failure.

Conclusions: In addition to providing a comprehensive list of risk factors and their relative importance, the study presents a major contribution by unifying the literature on information systems and medical infor - matics. Our checklist provides a basis for further research that may help practitioners identify the effective countermeasures for mitigating risks associated with the implementation of clinical information systems.

 
  • References

  • 1 Ferratt TW, Lederer AL, Hall SR, Krella JM. Surmounting Health Information Network Barriers. The Greater Dayton Area Experience, Health Care Manage Rev 1998; 23 (01) 70-76.
  • 2 Payton FC, Ginzberg MJ. Inter-organizational Health Care Systems Implementations. An Exploratory Study of Early Electronic Commerce Initiatives, Health Care Manage Rev 2001; 26 (02) 20-32.
  • 3 Paré G, Jaana M, Sicotte C. Systematic Review of Home Telemonitoring for Chronic Diseases. The Evidence Base, J Am Med Inform Assoc 2007; 14 (03) 269-277.
  • 4 Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG. Systematic Review: Impact of Health Information Technology on quality, efficiency, and costs of Medical Care. Ann Intern Med 2006; 144: E12-E22.
  • 5 Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled Trial of Direct Physician Order Entry: Effects on Physicians’ Time Utilization in Ambulatory Care Internal Medicine Practices. J Am Med Inform Assoc 2001; 8 (04) 361-371.
  • 6 Lepanto L, Paré G, Aubry D, Robillard P, Lesage J. Impact of PACS on Dictation Turnaround Time and Productivity. J Digit Imaging 2006; 19 (01) 92-97.
  • 7 Kuhn KA, Wurst SH, Bott OJ, Giuse DA. Expanding the Scope of Health Information Systems. Challenges and Developments. Methods Inf Med 2006; 45 (01) 43-52.
  • 8 Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Lloyd JF, Burke JP. A Computer-Assisted Management Program for Antibiotics and Other Antiinfective Agents. N Engl J Med 1998; 338 (04) 232-238.
  • 9 Michalowski W, Slowinski R, Wilk S, Farion KJ, Pike J, Rubin S. Design and Development of a Mobile System for Supporting Emergency Triage. Methods Inf Med 2005; 44 (01) 14-24.
  • 10 Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The Extent and Importance of Unintended Consequences Related to Computerized Provider Entry. J Am Med Inform Assoc 2007; 14 (04) 415-423.
  • 11 O’ Connell RT, Cho C, Shah N, Brown K, Shiffman RN. Take Note(s): Differential EHR Satisfaction with Two Implementations under One Roof. J Am Med Inform Assoc 2004; 11 (01) 43-49.
  • 12 Kleinke JD. Dot-Gov: Market Failure and the Creation of a National Health Information Technology System. Health Aff (Millwood) 2005; 24 (05) 1246-1262.
  • 13 Sicotte C, Paré G, Moreault MP, Paccioni A. A Risk Assessment of Two Interorganizational Clinical Information Systems. J Am Med Inform Assoc 2006; 13 (05) 557-566.
  • 14 Kaplan B. Culture Counts: How Institutional Values Affect Computer Use. M D Comput 2000; 17 (01) 23-26.
  • 15 Ozdas A, Miller RA. Care Provider Order Entry (CPOE): A Perspective on Factors Leading to Success or to Failure. Methods Inf Med 2007; 46 (01) 128-137.
  • 16 Aarts J, Berg M. Same Systems, Different Outcomes – Comparing the Implementation of Computerized Physician Order Entry in two Dutch Hospitals. Methods Inf Med 2006; 45 (01) 53-61.
  • 17 Lapointe L, Rivard S. Getting Physicians to Accept New Information Technology: Insights from Case Studies. CMAJ 2006; 174 (11) 1573-1578.
  • 18 Sicotte C, Denis JL, Lehoux P. The Computer Based Patient Record: A Strategic Issue in Process Innovation. J Med Syst 1998; 22 (06) 431-443.
  • 19 Paré G, Trudel MC. Knowledge Barriers to PACS Adoption and Implementation in Hospitals. Int J Med Inform 2007; 76 (01) 22-33.
  • 20 Schmidt R, Lyytinen K, Keil M, Cule P. Identifying Software Project Risks: An International Delphi Study. J Manage Inform Syst 2001; 17 (04) 5-36.
  • 21 Chiang MF, Starren JB. Software Engineering Risk Factors in the Implementation of a Small Electronic Medical Record System: The Problem of Scalability. Proc AMIA Symp 2002 pp 145-149.
  • 22 Brender J, Ammenwerth E, Nykanen P, Talmon J. Factors Influencing Success and Failure of Health Informatics Systems. Methods Inf Med 2006; 45 (01) 125-136.
  • 23 Barki H, Rivard S, Talbot J. Toward an Assessment of Software Development Risk. J Manage Inform Syst 1993; 10 (02) 203-225.
  • 24 Sherer SA, Alter S. Information System Risks and Risk Factors: Are they Mostly about Information Systems?. Commun AIS 2004; 14: 29-64.
  • 25 Doolan DF, Bates DW, James BC. The Use of Computers for Clinical Care: A Case Series of Advanced U S. Sites. J Am Med Inform Assoc 2003; 10 (01) 94-107.
  • 26 Starr P. Smart Technology, Stunted Policy: Developing Health Information Networks. Health Aff (Millwood) 1997; 16 (03) 91-105.
  • 27 Overhage JM, Evans L, Marchibroda J. Communities’ Readiness for Health Information Exchange: The National Landscape in 2004. J Am Med Inform Assoc 2005; 12 (02) 107-112.
  • 28 Markus ML. Power, Politics, and MIS Implementation. Commun ACM 1983; 26 (06) 430-444.
  • 29 Lyytinen K, Hirschheim R. Information Systems Failures: A Survey and Classification of the Empirical Literature. Oxford Surv Inform Tech 1987; 4: 257-309.
  • 30 Battacherjee A. Understanding Information Systems Continuance: An Expectation-Confirmation Model. MIS Quart 2001; 25 (03) 351-370.
  • 31 Miller RH, Sim I. Physicians’ Use of Electronic Medical Records: Barriers and Solutions. Health Aff (Millwood) 2004; 23 (02) 116-126.
  • 32 Spil TAM, Schuring RW, Michel-Verkerke MB. Electronic Prescription System: Do Professionals Use It?. Int J Healthc Technol Manage 2004; 6 (01) 32-55.
  • 33 Lee F, Teich JM, Spurr CD, Bates DW. Implementation of Physician Order Entry: User Satisfaction and Self-reported Usage Patterns. J Am Med Inform Assoc 1996; 3: 42-55.
  • 34 Ahmad A, Teater P, Bentley TD, Kuehn L, Kumar RR, Thomas A, Mekhjian HS. Key Attributes of a Successful Physician Order Entry System Implementation in a Multi-hospital Environment. J Am Med Inform Assoc 2002; 9: 16-24.
  • 35 Van Der Meijden MJ, Tange HJ, Troost J, Hasman A. Determinants of Success of Inpatient Clinical Information Systems: A Literature Review. J Am Med Inform Assoc 2003; 10 (03) 235-243.
  • 36 Siau K. Inter-organizational Systems and Competitive Advantages – Lessons from History. J Comput Inform Syst 2003; 44 (01) 33-39.
  • 37 Anderson JG, Aydin CE. Evaluating the Impact of Health Care Information Systems. Int J Technol Assess Health Care 1997; 13 (02) 380-393.
  • 38 Hunsucker JL, Loos D. Transition Management: An Analysis of Strategic Considerations for Effective Implementation. Eng Manage Int 1989; 5: 167-178.
  • 39 Tiwana A, Keil M. The One-Minute Risk Assessment Tool. Commun ACM 2004; 47 (11) 73-77.
  • 40 Jarvenpaa SL, Ives B. Executive Involvement and Participation in the Management of Information Technology. MIS Quart 1991; 15 (02) 205-227.
  • 41 Hazeltine FW. The Key to Successful Implementations. Prod Inv Manage R APICS NEWS 1990; 10 (11) 41-42 45.
  • 42 Schmidt R. Managing Delphi Surveys Using Nonparametric Statistical Techniques. Decision Sci 1997; 28 (03) 763-774.
  • 43 Beath CM. Supporting the Information Technology Champion. MIS Quart 1991; 15 (03) 355-372.
  • 44 Dong L, Sun H, Fang Y. Do Perceived Leadership Behaviors Affect User Technology Beliefs? An Examination of the Impact of Project Champions and Direct Managers. Commun AIS 2007; 19: 655-664.
  • 45 Mintzberg H. Structure et Dynamique des Organisations. Paris: Editions d’Organisation; 1982
  • 46 Paré G, Elam JJ, Ward CG. Implementation of a Patient Charting System: Challenges Encountered and Tactics Adopted in a Burn Center. J Med Syst 1997; 21 (01) 49-66.
  • 47 Jeyeraj A, Rottman JW, Lacity MC. A Review of the Predictors, Linkages, and Biases in IT Innovation Adoption Research. J Inform Tech 2006; 21 (01) 1-23.
  • 48 Davis FD. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quart 1989; 13 (03) 318-339.
  • 49 Lorenzi NM, Riley RT. Organizational Aspects of Health Informatics. Managing Technological Change, New York, NY: Springer Verlag; 1995
  • 50 Kanter RM. The Enduring Skills of Change Leaders. Ivey Bus J 2000: 31-36.
  • 51 Kotter JP. Leading Change. Boston: Harvard University Press; 1996
  • 52 Shu K, Boyle D, Spurr C, Horsky J, Heiman H, O’ Connor P, Lepore J, Bates DW. Comparison of Time Spent Writing Orders on Paper with Computerized Physician Order Entry. Medinfo 2001; 10 Pt 2 1207-1211.
  • 53 Weiner M, Gress T, Thiemann DR, Jenckes M, Reel SI, Mandell SF, Bass EB. Contrasting Views of Physicians and Nurses about an Inpatient Computer- based Provider Order-entry System. J Am Med Inform Assoc 1999; 6 (03) 234-244.
  • 54 Chiasson M, Reddy M, Kaplan B, Davidson E. Expanding Multi-disciplinary Approaches to Healthcare Information Technologies: What does Information Systems Offer Medical Informatics?. Int J Med Inform 2007; 76S: S89-S97.