User Testing an Information Foraging Tool for Ambulatory Surgical Site Infection Surveillance

Dean J. Karavite; Matthew W. Miller; Mark J. Ramos; Susan L. Rettig; Rachael K. Ross; Rui Xiao; Naveen Muthu; A. Russell Localio; Jeffrey S. Gerber; Susan E. Coffin; Robert W. Grundmeier

doi:10.1055/s-0038-1675179

Applied Clinical Informatics, Inhaltsverzeichnis

Appl Clin Inform 2018; 09(04): 791-802
DOI: 10.1055/s-0038-1675179

Case Report

Georg Thieme Verlag KG Stuttgart · New York

User Testing an Information Foraging Tool for Ambulatory Surgical Site Infection Surveillance

Dean J. Karavite

¹Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

,

Matthew W. Miller

¹Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

,

Mark J. Ramos

¹Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

,

Susan L. Rettig

²Department of Infection Prevention and Control, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

,

Rachael K. Ross

³Division of Infectious Disease, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

,

Rui Xiao

⁴Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States

,

Naveen Muthu

¹Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

⁵ Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States

,

A. Russell Localio

⁴Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States

,

Jeffrey S. Gerber

³Division of Infectious Disease, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

⁵ Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States

,

Susan E. Coffin

³Division of Infectious Disease, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

⁵ Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States

,

Robert W. Grundmeier

¹Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

⁵ Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States

› Institutsangaben

Abstract

Background Surveillance for surgical site infections (SSIs) after ambulatory surgery in children requires a detailed manual chart review to assess criteria defined by the National Health and Safety Network (NHSN). Electronic health records (EHRs) impose an inefficient search process where infection preventionists must manually review every postsurgical encounter (< 30 days). Using text mining and business intelligence software, we developed an information foraging application, the SSI Workbench, to visually present which postsurgical encounters included SSI-related terms and synonyms, antibiotic, and culture orders.

Objective This article compares the Workbench and EHR on four dimensions: (1) effectiveness, (2) efficiency, (3) workload, and (4) usability.

Methods Comparative usability test of Workbench and EHR. Objective test metrics are time per case, encounters reviewed per case, time per encounter, and retrieval of information meeting NHSN definitions. Subjective measures are cognitive load using the National Aeronautics and Space Administration (NASA) Task Load Index (NASA TLX), and a questionnaire on system usability and utility.

Results Eight infection preventionists participated in the test. There was no difference in effectiveness as subjects retrieved information from all cases, using both systems, to meet the NHSN criteria. There was no difference in efficiency in time per case between the Workbench and EHR (8.58 vs. 7.39 minutes, p = 0.36). However, with the Workbench subjects opened fewer encounters per case (3.0 vs. 7.5, p = 0.002), spent more time per encounter (2.23 vs. 0.92 minutes, p = 0.002), rated the Workbench lower in cognitive load (NASA TLX, 24 vs. 33, p = 0.02), and significantly higher in measures of usability.

Conclusion Compared with the EHR, the Workbench was more usable, short, and reduced cognitive load. In overall efficiency, the Workbench did not save time, but demonstrated a shift from between-encounter foraging to within-encounter foraging and was rated as significantly more efficient. Our results suggest that infection surveillance can be better supported by systems applying information foraging theory.

Keywords

electronic health records and systems - patient records - specific types - clinical information systems - surgical wound infection - ambulatory surgical procedures - medical informatics - decision making - computer-assisted - user–computer interface

Volltext

Referenzen

References
1 Klevens RM, Edwards JR, Richards Jr CL. , et al. Estimating health care-associated infections and deaths in U.S. hospitals, 2002. Public Health Rep 2007; 122 (02) 160-166
2 Ambulatory Surgery Centers. A Positive Trend in Healthcare. Ambulatory Surgical Center Coalition. Available at: http://www.ascassociation.org . Accessed 2017
3 Haley RW. The scientific basis for using surveillance and risk factor data to reduce nosocomial infection rates. J Hosp Infect 1995; 30 (Suppl): 3-14
4 Consensus paper on the surveillance of surgical wound infections. The Society for Hospital Epidemiology of America; The Association for Practitioners in Infection Control; The Centers for Disease Control; The Surgical Infection Society. Infect Control Hosp Epidemiol 1992; 13 (10) 599-605
5 Surgical Site Infection (SSI) Event, Procedure-associated model, Center for Disease Control. Available at: https://www.cdc.gov/nhsn/pdfs/pscmanual/9pscssicurrent.pdf . Accessed 2017
6 Grammatico-Guillon L, Baron S, Gaborit C, Rusch E, Astagneau P. Quality assessment of hospital discharge database for routine surveillance of hip and knee arthroplasty-related infections. Infect Control Hosp Epidemiol 2014; 35 (06) 646-651
7 van Mourik MS, Troelstra A, van Solinge WW, Moons KG, Bonten MJ. Automated surveillance for healthcare-associated infections: opportunities for improvement. Clin Infect Dis 2013; 57 (01) 85-93
8 Woeltje KF, Lin MY, Klompas M, Wright MO, Zuccotti G, Trick WE. Data requirements for electronic surveillance of healthcare-associated infections. Infect Control Hosp Epidemiol 2014; 35 (09) 1083-1091
9 Pirolli P, Card S. Information foraging. Psychol Rev 1999; 106 (04) 643
10 Schraagen JM, Chipman SF, Shalin VL. Cognitive Task Analysis. Psychology Press; 2000
11 Paas F, Renkl A, Sweller J. Cognitive load theory and instructional design: recent developments. Educ Psychol 2003; 38 (01) 1-4
12 Chi EH, Pirolli P, Chen K. , et al. Using information scent to model user information needs and actions and the Web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2001 :490–497
13 Card SK, Pirolli P, Van Der Wege M. , et al. Information scent as a driver of Web behavior graphs: results of a protocol analysis method for Web usability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2001 :498–505
14 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
15 Nielsen J. Usability inspection methods. In Conference Companion on Human Factors in Computing Systems. ACM; 1999: 413-414
16 Rubin J, Chisnell D. Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests. Indianapolis, IN: John Wiley & Sons; 2008
17 Grundmeier RW, Xiao R, Ross RK. , et al. Identifying surgical site infections in electronic health data using predictive models. J Am Med Inform Assoc 2018; 25 (09) 1160-1166
18 Cox GM, Cochran WG. Experimental Designs. New York: Wiley; 1957
19 Ericsson K, Simon H. Verbal reports as data. Psychol Rev 1980; 87 (03) 215-251
20 Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv Psychol 1988; 52: 139-183
21 Hart SG. NASA-task load index (NASA-TLX); 20 years later. Proc Hum Factors Ergon Soc Annu Meet 2006; 50 (09) 904-908
22 R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing;2017. Available at: http://www.r-project.org/ . Accessed March 2, 2018
23 Frøkjær E, Hertzum M, Hornbæk K. . Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.ACM;2000 April 1:345–352
24 Sweller John. Cognitive load theory, learning difficulty, and instructional design. Learning Instruct 1994; 4 (04) 295-312
25 Ahmed A, Chandra S, Herasevich V, Gajic O, Pickering BW. The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Crit Care Med 2011; 39 (07) 1626-1634
26 Plaisant C, Milash B, Rose A. , et al. LifeLines: visualizing personal histories. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM;1996:221–227
27 Card SK, Mackinlay JD, Schneiderman B. Information Visualization: Using Vision to Think. San Francisco, CA: Morgan-Kaufmann; 1999