Download PDF
Appl Clin Inform 2019; 10(02): 237-246
DOI: 10.1055/s-0039-1683985
Research Article
Georg Thieme Verlag KG Stuttgart · New York

Assessing the Safety of Custom Web-Based Clinical Decision Support Systems in Electronic Health Records: A Case Study

Jeritt G. Thayer
1   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
,
Jeffrey M. Miller
1   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
,
Alexander G. Fiks
1   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
2   Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States
,
Linda Tague
3   Department of Information Services, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
,
Robert W. Grundmeier
1   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
2   Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States
› Author Affiliations Funding Dr. Fiks is a coinventor of the “Care Assistant” software that was evaluated in this study. He holds no patent on the software and has earned no money from this invention. No licensing agreement exists. Dr. Fiks has also received an Independent Research Grant from Pfizer from which he personally drew no support. Dr. Grundmeier is a coinventor of the “Care Assistant” software that was evaluated in this study. He holds no patent on the software and has earned no money from this invention. No licensing agreement exists. Mr. Miller is a coinventor of “Project SHARE” which is a Care Assistant module to support the care of children with attention deficit disorder. He holds no patent on the software and has earned no money from this invention. No licensing agreement exists. The other authors have indicated they have no financial relationships relevant to this article to disclose.
Further Information

Publication History

29 November 2018

13 February 2019

Publication Date:
03 April 2019 (online)

   Permissions and Reprints

Abstract

Background With the widespread adoption of vendor-supplied electronic health record (EHR) systems, clinical decision support (CDS) customization efforts beyond those anticipated by the vendor may require the use of technologies external to the EHR such as web services. Pursuing such customizations, however, is not without risk. Validating the expected behavior of a customized CDS system in the high-volume, complex environment of the live EHR is a challenging problem.

Objective This article identifies technology failures that impacted clinical care related to web service-based advanced custom CDS systems embedded in the complex sociotechnical context of a production EHR.

Methods In an academic health system’s primary care network, we performed an inventory of incidents between January 1, 2008 and December 31, 2016 related to a customized CDS system and performed a targeted review of changes in the CDS source code. Additional feedback on the root cause of individual incidents was obtained through interviews with members of the CDS project teams.

Results We identified five CDS malfunctions that impaired clinical workflow. The mechanisms for these failures are mapped to four characteristics of well-behaved applications: (1) system integrity; (2) data integrity; (3) reliability; and (4) scalability. Over the 9-year period, two malfunctions of the customized CDS significantly impaired clinical workflow for a total of 5 hours. Lesser impacts—loss of individual features with straightforward workarounds—arose from three malfunctions, which affected users on 53 days.

Discussion Advanced customization of EHRs for the purpose of CDS can present significant risks to clinical workflow.

Conclusion This case study highlights that advanced customization of CDS within a commercial EHR may support care for complex patient populations, but ongoing monitoring and support is required to ensure its safe use.

Keywords

clinical decision support - electronic health records and systems - clinical information systems - safety - error management

Protection of Human and Animal Subjects

This study did not involve human or animal subjects research.


 

  • References

  • 1 Sim I, Gorman P, Greenes RA. , et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc 2001; 8 (06) 527-534
    CrossrefPubMedGoogle Scholar
  • 2 Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med 1994; 120 (02) 135
    PubMedGoogle Scholar
  • 3 Kaplan B. Evaluating informatics applications--clinical decision support systems literature review. Int J Med Inform 2001; 64 (01) 15-37
    CrossrefPubMedGoogle Scholar
  • 4 Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians. Arch Intern Med 2000; 160 (03) 301-308
    CrossrefPubMedGoogle Scholar
  • 5 Gardner RM, Lundsgaarde HP. Evaluation of user acceptance of a clinical expert system. J Am Med Inform Assoc 1994; 1 (06) 428-438
    CrossrefPubMedGoogle Scholar
  • 6 Eligible Professional Meaningful Use Core Measures Measure 6 of 17, Stage 2, Clinical Decision Support Rule. Available at: https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/Stage2_EPCore_6_ClinicalDecisionSupport.pdf . Accessed July 12, 2017
    PubMed
  • 7 Definition of Terms Eligible Professional Meaningful Use Core Measures Measure 10 of 13. Available at: https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/11_Clinical_Decision_Support_Rule.pdf . Accessed July 12, 2017
    PubMed
  • 8 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; 11 (02) 104-112
    PubMedGoogle Scholar
  • 9 Wright A, Hickman T-TT, McEvoy D. , et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
    CrossrefPubMedGoogle Scholar
  • 10 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; 13 (05) 547-556
    CrossrefPubMedGoogle Scholar
  • 11 Koppel R, Metlay JP, Cohen A. , et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-1203
    CrossrefPubMedGoogle Scholar
  • 12 Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007; 14 (04) 415-423
    CrossrefPubMedGoogle Scholar
  • 13 Wright A, Ai A, Ash J. , et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (05) 496-506
    CrossrefPubMedGoogle Scholar
  • 14 Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8 (03) 910-923
    Article in Thieme ConnectPubMedGoogle Scholar
  • 15 Mandl KD, Mandel JC, Murphy SN. , et al. The SMART Platform: early experience enabling substitutable applications for electronic health records. J Am Med Inform Assoc 2012; 19 (04) 597-603
    CrossrefPubMedGoogle Scholar
  • 16 Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016; 23 (05) 899-908
    CrossrefPubMedGoogle Scholar
  • 17 Mandl KD, Kohane IS. No small change for the health information economy. N Engl J Med 2009; 360 (13) 1278-1281
    CrossrefPubMedGoogle Scholar
  • 18 App Orchard Program Details. 2017 . Available at: http://www.epic.com/ . Accessed October 9, 2017
    PubMed
  • 19 Gorton I. Software quality attributes. In: Essential Software Architecture. Berlin, Heidelberg: Springer Berlin Heidelberg; 2011: 23-38 . Available at: http://link.springer.com/10.1007/978-3-642-19176-3_3 . Accessed October 9, 2017
    CrossrefGoogle Scholar
  • 20 Fiks AG, Grundmeier RW, Margolis B. , et al. Comparative effectiveness research using the electronic medical record: an emerging area of investigation in pediatric primary care. J Pediatr 2012; 160 (05) 719-724
    CrossrefPubMedGoogle Scholar
  • 21 Overview - FHIR v3.0.1. Available at: https://www.hl7.org/fhir/overview.html . Accessed July 13, 2017
    PubMed
  • 22 Encounter - FHIR v3.0.1. Available at: https://www.hl7.org/fhir/encounter.html . Accessed August 29, 2017
    PubMed
  • 23 Gordon WJ, Baronas J, Lane WJ. A FHIR human leukocyte antigen (HLA) interface for platelet transfusion support. Appl Clin Inform 2017; 8 (02) 603-611
    Article in Thieme ConnectPubMedGoogle Scholar
  • 24 Bell LM, Grundmeier R, Localio R. , et al. Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics 2010; 125 (04) e770-e777
    CrossrefPubMedGoogle Scholar
  • 25 Utidjian LH, Hogan A, Michel J. , et al. Clinical decision support and palivizumab: a means to protect from respiratory syncytial virus. Appl Clin Inform 2015; 6 (04) 769-784
    Article in Thieme ConnectPubMedGoogle Scholar
  • 26 Fiks AG, Mayne S, Hughes CC. , et al. Development of an instrument to measure parents' preferences and goals for the treatment of attention deficit-hyperactivity disorder. Acad Pediatr 2012; 12 (05) 445-455
    CrossrefPubMedGoogle Scholar
  • 27 Fiks AG, Mayne SL, Karavite DJ. , et al. Parent-reported outcomes of a shared decision-making portal in asthma: a practice-based RCT. Pediatrics 2015; 135 (04) e965-e973
    CrossrefPubMedGoogle Scholar
  • 28 Fiks AG, Mayne S, Karavite DJ, DeBartolo E, Grundmeier RW. A shared e-decision support portal for pediatric asthma. J Ambul Care Manage 2014; 37 (02) 120-126
    CrossrefPubMedGoogle Scholar
  • 29 Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics 2007; 120 (04) 707-714
    CrossrefPubMedGoogle Scholar
  • 30 Fiks AG, Grundmeier RW, Mayne S. , et al. Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt. Pediatrics 2013; 131 (06) 1114-1124
    CrossrefPubMedGoogle Scholar
  • 31 Forrest CB, Fiks AG, Bailey LC. , et al. Improving adherence to otitis media guidelines with clinical decision support and physician feedback. Pediatrics 2013; 131 (04) e1071-e1081
    CrossrefPubMedGoogle Scholar
  • 32 Lipman TH, Cousounis P, Grundmeier RW. , et al. Electronic health record mid-parental height auto-calculator for growth assessment in primary care. Clin Pediatr (Phila) 2016; 55 (12) 1100-1106
    CrossrefPubMedGoogle Scholar
  • 33 Asthma Clinical Pathway — Inpatient | Children's Hospital of Philadelphia. Available at: http://www.chop.edu/clinical-pathway/asthma-inpatient-care-clinical-pathway . Accessed July 12, 2017
    PubMed
  • 34 Sittig DF, Singh H. Defining health information technology-related errors: new developments since to err is human. Arch Intern Med 2011; 171 (14) 1281-1284
    CrossrefPubMedGoogle Scholar
  • 35 Wickens CD, Hollands JG, Banbury S, Parasuraman R. Engineering Psychology and Human Performance. Vol. 4, Engineering psychology and human performance. Pearson; 2013: 394-395
    Google Scholar
  • 36 Breznitz S. Cry-wolf: The Psychology of False Alarms. Hillsdale, NJ: Lawrence Erlbaum Associates; 1984: 9-14
    Google Scholar
  • 37 Sorkin RD. Why are people turning off our alarms?. J Acoust Soc Am 1988; 84 (03) 1107-1108
    CrossrefPubMedGoogle Scholar
  • 38 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
    Article in Thieme ConnectPubMedGoogle Scholar
  • 39 Drews FA. The frequency and impact of task interruptions in the ICU. Proc Hum Factors Ergon Soc Annu Meet 2007; 51 (11) 683-686
    CrossrefPubMedGoogle Scholar
  • 40 Wiegmann DA, ElBardissi AW, Dearani JA, Daly RC, Sundt III TM. Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery 2007; 142 (05) 658-665
    CrossrefPubMedGoogle Scholar
  • 41 Sevdalis N, Forrest D, Undre S, Darzi A, Vincent C. Annoyances, disruptions, and interruptions in surgery: the Disruptions in Surgery Index (DiSI). World J Surg 2008; 32 (08) 1643-1650
    CrossrefPubMedGoogle Scholar
  • 42 Horsky J, Phansalkar S, Desai A, Bell D, Middleton B. Design of decision support interventions for medication prescribing. Int J Med Inform 2013; 82 (06) 492-503
    CrossrefPubMedGoogle Scholar
  • 43 Gaba DM, Howard SK, Small SD. Situation awareness in anesthesiology. Hum Factors 1995; 37 (01) 20-31
    CrossrefPubMedGoogle Scholar
  • 44 Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012; 45 (06) 1202-1216
    CrossrefPubMedGoogle Scholar
  • 45 Walker JM, Carayon P, Leveson N. , et al. EHR safety: the way forward to safe and effective systems. J Am Med Inform Assoc 2008; 15 (03) 272-277
    CrossrefPubMedGoogle Scholar
  • 46 Sittig DF, Classen DC. Safe electronic health record use requires a comprehensive monitoring and evaluation framework. JAMA 2010; 303 (05) 450-451
    CrossrefPubMedGoogle Scholar
  • 47 Sittig DF, Singh H. Defining health information technology-related errors: new developments since to err is human. Arch Intern Med 2011; 171 (14) 1281-1284
    CrossrefPubMedGoogle Scholar
  • 48 Zheng K, Haftel HM, Hirschl RB, O'Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inform Assoc 2010; 17 (04) 454-461
    CrossrefPubMedGoogle Scholar