User-Centered Design in Pediatric Acute Care Settings Antimicrobial Stewardship

Abstract

Background Antibiotic prescribing in ambulatory care centers is increasing. Previous research suggests that 20 to 50% of antibiotic prescriptions are either unnecessary or inappropriate. Unnecessary antibiotic consumption can harm patients by increasing antibiotic resistance and drug-associated toxicities, and the reasons for such use are multifactorial. Antimicrobial Stewardship Programs (ASP) were developed to guide better use of antibiotics. A core element of ASP is to provide feedback to clinical providers. To create clinically meaningful feedback, user-center design (UCD) is a robust approach to include end-users in the design process to improve systems.

Objective The study aimed to take a UCD approach to developing antibiotic prescribing feedback through input from clinicians in two ambulatory care settings.

Methods We conducted two group prototyping sessions with pediatric clinicians who practice in the emergency department and urgent care settings at a tertiary care children's hospital. Participants received background on the problem of antibiotic prescribing and then were interviewed about their information needs, perceived value, and desired incentives for a prescribing feedback system. Sessions concluded with their response and recommendations to sample sections of an antibiotic feedback report including orienting material, report detail, targeted education, and resources.

Results A UCD approach was found to be highly valuable in the development of a feedback mechanism that is viewed as desirable by clinicians. Clinicians preferred interpreting the data themselves with aids such as diagrams and charts over the researcher concluded statements about the clinician's behavior. Specific feedback that clinicians considered redundant were removed from the model if preexisting alerts were established.

Conclusion Integrating a UCD approach in developing ASP feedback identified desirable report characteristics that substantially modified preliminary wireframes for feedback. Future research will evaluate the clinical effectiveness of our feedback reports in outpatient settings.

Keywords

user-centered design - ambulatory care - antimicrobial stewardship programs - feedback - pediatrics - requirements analysis and design - qualitative

Background and Significance

In 2015, 269 million antibiotics were prescribed in outpatient care settings,[1] which represents more than 80% of all antibiotic use.[2] An estimated 30% of these antibiotics were prescribed unnecessarily.[2] Antibiotic prescribing for respiratory diagnoses that do not require antibiotic therapy is the highest in urgent care and emergency department (ED) settings.[3] Unnecessary antibiotics are a contributor to antibiotic resistance and can result in adverse drug events that frequently lead to acute care visits.[4]

To guide more appropriate use of antibiotics and reduce unnecessary prescribing in health care, antimicrobial stewardship programs (ASPs) are recommended by the Centers for Disease Control and Prevention (CDC).[5] Despite the high prevalence of antibiotic use and subsequent inappropriate use in outpatient settings, ASPs have historically been focused on inpatient settings.[5] [6]

Development of effective ASPs involves the application of ASP techniques. Web-based programs can enhance the reporting and communication of antimicrobial stewardship that are both cost efficient while being perceived positively by users.[7] [8] [9] The other research has developed clinical decision support for ASPs; however, these lack a longitudinal feedback mechanism for providers.[10] [11] [12] A core component of successful ASPs, as recommended by the CDC, includes audit and feedback.[13] A Cochrane review of audit and feedback demonstrated small important improvements in provider behavior,[14] but rarely occurs in acute care settings such as the ED.[15] Personalization coupled with audit and feedback methods from ASP enhances the acceptance of feedback.[15] [16] Unfortunately, personalization is infrequent, hard to accomplish, and challenging to systematize.[17]

Strategies and techniques are needed to address the personalization challenge to adapt ASPs for outpatient settings where the patient population and severity of illness differ from inpatient settings, and the number of patients cared for by providers may be much higher.[10] User-centered design (UCD) is a technique from human factors engineering that offers a systematic approach to addressing personalization for both design visualizations and facilitating interaction with the end user product.[18] UCD includes the user in the design of the interface and throughout the design process.[19] Finally, UCD improves the end user experience and when used, provides benefits in the dissemination and implementation of programs involving choice.[20] [Fig. 1] depicts the UCD approach where the design is iteratively built based on feedback of previous versions of the design and input from participants. The process continues in a loop of design, input on interface changes, and redesign. This continues until a final product is selected that is derived from multiple meetings/input sessions.

Fig. 1 User-centered Design Approach to Technology development. Reprinted with permission from Matt Weinger, copyright 2017.

Objective

The goal of this project was to take a UCD approach to develop a user interface for delivery of feedback to clinicians in outpatient care settings with high antibiotic prescription rates, a variety of potential prescribers, and potentially inappropriate prescribing practices that could benefit from the implementation of ASP initiatives. This would assist the clinicians in understanding and changing their clinical practice to be more in line with current recommended ASP guidance.

Methods

Participants and Setting

This study was conducted at a tertiary care medical center in Nashville, Tennessee. The medical center serves as specialty and primary care for over 2 million adult and children patients every year in inpatient and outpatient settings throughout Middle Tennessee. The state of Tennessee is an exemplar for the need for ASPs; there were 1,098 antibiotic prescriptions written for every 1,000 pediatric patients in 2017.[1] This is 50% higher than the national U.S. average. This study focused on pediatric urgent care outpatient practices; hence, our participants were limited to prescribing clinicians from the pediatric After-Hours Clinics (AHC) and the ED. We contacted the medical directors from both settings to participate and assist in participant recruitment. We excluded trainees as they do not work in our AHC setting and only rotate through the ED for 1 to 2 months during each year of training. Design sessions with participants also included investigators who have backgrounds in emergency medicine, infectious disease, informatics, and human factors engineering. We used a snowball sampling approach and asked that the medical directors recruit staff interested in participating. We also conducted individual interviews with additional interested participants between design sessions. The study was performed in compliance with the World Medical Association Declaration of Helsinki on ethical principles for medical research involving human subjects and was approved by Vanderbilt University Institutional Review Board.

Clinical Use Case

In an effort to implement antibiotic stewardship in an acute care setting, we sought to identify a clinical condition in which there was relatively clear guidance regarding optimal antibiotic prescribing with high variability in practice patterns. We reviewed the most common conditions seen in pediatric ED and AHC settings between November 2017 and March 2019 including otitis, bronchitis, pneumonia, sinusitis, skin and soft tissue infections, and urinary tract infection (UTI). Pediatric UTI was the fourth most common condition but had the most practice variability and use of broad-spectrum antibiotics. Concurrent with the development of this wireframe, our ASP group was developing local guidelines for UTI treatment, with plans to recommend cephalexin and nitrofurantoin as first-line choices based on local urine culture and organism susceptibility data. The baseline rate of cephalexin and nitrofurantoin use for UTI was 57% in the AHC and 31% in the ED. Thus, our group reached consensus to select pediatric UTI as the targeted condition for this project.

Wireframe Development

Wireframe development was guided by the UCD Approach ([Fig. 1]). We planned to conduct two prototyping sessions with clinicians in the ED and AHC to discuss pediatric UTI antibiotic prescribing. Our first session was conducted with two pediatric emergency physicians, and the second session was conducted with six AHC physicians. Between sessions, our wireframe was revised based upon feedback obtained during the first session. As an investigator group if we unanimously agreed that we were approaching theoretical saturation, the point at which no substantial new information was obtained, we would move to one-on-one interviews to attempt to identify feedback not elicited during the larger group sessions. If saturation was not achieved, we planned to continue our prototyping sessions until saturation was achieved. Prototyping sessions were planned as 1-hour meetings and organized to provide background by content experts (i.e., pediatric infectious disease), user requirements ([Table 1]), and formative evaluation of a medium-fidelity wireframe as well as several different design alternatives during these sessions.

Table 1 User specifications

Information needs	• What types of feedback do you currently receive about the care you deliver in the emergency department?
	• What information about your antibiotic prescribing practices would you want to know?
Timing and delivery	• How often would you want feedback about your patients?
	• How would you like to receive the report?
Practice and sustainability	• What information would motivate you to change your prescription habits?
Incentives	• In addition to personal interest, is there anything that might make you/your colleagues more motivated to review and use this feedback?
Targeted education and resources	• What information would you like for additional support and resources?

A medium-fidelity wireframe was developed using the investigators' combined experience as a board-certified, practicing emergency physician, human factors engineer, and two pediatric infectious disease physicians. The initial wireframe can be seen in [Fig. 2].

Fig. 2 Initial antibiotic feedback wireframe.

Results

The study received feedback from 11 clinicians. Eight physicians participated in prototype sessions (two EDs and six AHCs) and three ED physicians were interviewed one on one. All 11 were attending physicians in their respective clinical settings.

Understanding Context

Clinicians from both settings reported currently receiving minimal feedback about their clinical practice, which primarily involved patient “bounce-back” or return rates to the hospital after 48 hours in the ED and 72 hours in the AHC. Clinicians in both settings reported that these reports lacked context and were merely a notification prompting the clinician to review the patient's chart to identify what happened. In particular, the ED clinicians said that they received these notifications from multiple sources and up to three times per return patient. The required effort of searching for the information often exceeded the perceived benefit. Information on these patient visits was ignored in instances where the clinicians found little to no benefit such as bounce-back notifications.

User Requirements

Information Content

A requested feature of the interface was a visual representation of their UTI antibiotic prescribing patterns along with a summarized description and interpretation of the findings. In other words, to describe what the reported statistic means in plain language. Moreover, the clinicians requested the information have informative details, but exclude positive or negative connotation allowing the provider to judge for themselves and modify their practice accordingly. Clinicians favored a display that provided details about practice and whether their practice percentages conformed with accepted national and regional standards. Clinicians desired a blinded peer comparison with other providers from similar settings such as urgent care clinics, primary care clinics, and EDs. They also preferred a neutral to positive feeling about their performance that was constructive rather than, as one participant noted, a “screen full of feel bads.” Clinicians also desired information about the cost of prescribed antibiotics, effectiveness of the prescription choice (e.g., whether the organisms that grew in culture were susceptible to the antibiotics given), and about the distribution of antibiotics prescribed for a specific condition by their peers.

Timing and Delivery

Clinicians favored models where the prescribing report was passively delivered to the clinician on a quarterly basis with the option to interact with the feedback report in real-time when an adverse condition such as bounce-back occurred. Monthly or more frequent intervals for delivery were felt to have too small of a sample size to be sufficient for clinicians to derive any benefit. Users also requested that the reports be available to view on a smartphone. While most clinicians preferred e-mail delivery, there was consensus that a user-defined delivery mechanism through e-mail, text, or paper would be most desirable.

Final Wireframe

The resulting wireframes can be seen in [Figs. 3] and [4]. This design was achieved following the two prototyping sessions and three design consultation with individual clinicians. As can be seen, there are different presentations of the information that the clinician would be able to scroll and see about their prescribing practice.

Fig. 3 Antibiotic feedback wireframe concept after iterations with clinicians focused on cost information.

Fig. 4 Antibiotic feedback wireframe concept after iterations with clinicians focused on peer comparison information.

Discussion

We used an iterative and UCD approach to identify desirable characteristics for automated antimicrobial stewardship feedback to enhance use and interaction among outpatient acute care physicians through a pediatric UTI use case. We were able to develop a wireframe that met user specifications. Similar among clinicians in both the ED and AHC, clinicians desired information that communicated summarized, comparable performance, cost, and efficacy in a simple and constructive format. Minor differences between the two groups existed when extending use beyond pediatric UTIs and involved the prevalence of illnesses in the provider's clinical setting (e.g., pneumonia vs. cellulitis). Clinician preferences for the timeliness of information delivery is consistent with best practices for ASPs.[11]

A unique benefit of this approach is that researchers were able to better understand clinician prescribing practices. Mainly, these insights revolved around learning about why clinicians prescribed the way they did and additional obstacles that they faced when prescribing. For example, during one of our prototyping sessions clinicians spoke about how parents would refuse to let their child be prescribed antibiotics that were taken more than one time a day. Alternatively, parents also refused various antibiotics because they were pills rather than liquid. In this case, one physician recommended that they would send the antibiotic prescription to a compounding pharmacy that could grind the medication and make it into a flavored suspension for pediatric patients.

Future work will incorporate user-defined specifications into antibiotic feedback reporting, will evaluate the effectiveness of the UCD approach feedback tool, and will describe feedback from final end users who did not participate in the initial project design, as well as details about patterns of use. Additionally, future work should evaluate the generalizability and degree of customization necessary within similar acute care settings (e.g., unscheduled urgent or emergency care) with scheduled settings, among different provider types (e.g., trainees, nurse practitioners, and physician assistants) and among different patient populations (e.g., pediatric vs. adult).

Most current ASP approaches lack UCD-designed feedback. One example during this study exists in the repetitive nature of bounce-back rates to the ED. While the project developed an interface to provide feedback, further studies should include aspects such as self-interpretation of data, adequate timing and delivery, and concise material with appropriate phrasing for both future models and in general practice for delivering feedback.

Conclusion

The feedback clinicians receive about antibiotic prescribing practices is limited in outpatient care. Clinicians were open to suggestions on practice standards if appropriate concerns around application were addressed.

Clinical Relevance Statement

There is limited research on how performance feedback information systems should provide clinician specific information and recommendations in a usable and timely way without negatively impacting clinic workflow. This case study can assist others in how the employ a UCD approach to system design. Finally, our observations may be used as a starting point for developing similar types of feedback systems within other organizations.

Multiple Choice Questions

1. Which of the following must you involve in the design process when developing a new application?

• Funders

• Potential users

• Vendors

• Patients

Correct Answer: The correct answer is option b.

2. What do potential users provide by doing interviews and prototyping sessions?

• Meaningful feedback about the design

• Functionality that is missing from the design

• Potential barriers to using the system

• All of the above

Correct Answer: The correct answer is option d.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on ethical principles for medical research involving human subjects and was reviewed by the Vanderbilt University Institutional Review Board.

• References

• 1 Control CDC. Prevention. Antibiotic use in the United States, 2017: progress and opportunities. Atlanta, GA: US Department of Health and Human Services, CDC; 2017
• 2 Fleming-Dutra KE, Hersh AL, Shapiro DJ. et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010–2011. JAMA 2016; 315 (17) 1864-1873
• 3 Palms DL, Hicks LA, Bartoces M. et al. Comparison of antibiotic prescribing in retain clinics, urgent care centers, emergency departments, and traditional ambulatory care settings in the United States. JAMA Intern Med 2018; 178 (09) 1267-1269
• 4 Shehab N, Patel PR, Srinivasan A, Budnitz DS. Emergency department visits for antibiotic-associated adverse events. Clin Infect Dis 2008; 47 (06) 735-743
• 5 Gerber JS, Prasad PA, Fiks AG. et al. Durability of benefits of an outpatient antimicrobial stewardship intervention after discontinuation of audit and feedback. JAMA 2014; 312 (23) 2569-2570
• 6 Pollack LA, Srinivasan A. Core elements of hospital antibiotic stewardship programs from the Centers for Disease Control and Prevention. Clin Infect Dis 2014; 59 (Suppl. 03) S97-S100
• 7 Agwu AL, Lee CK, Jain SK. et al. A World Wide Web-based antimicrobial stewardship program improves efficiency, communication, and user satisfaction and reduces cost in a tertiary care pediatric medical center. Clin Infect Dis 2008; 47 (06) 747-753
• 8 Sick AC, Lehmann CU, Tamma PD, Lee CK, Agwu AL. Sustained savings from a longitudinal cost analysis of an internet-based preapproval antimicrobial stewardship program. Infect Control Hosp Epidemiol 2013; 34 (06) 573-580
• 9 Venugopal V, Lehmann CU, Diener-West M, Agwu AL. Longitudinal evaluation of a World Wide Web-based antimicrobial stewardship program: assessing factors associated with approval patterns and trends over time. Am J Infect Control 2014; 42 (02) 100-105
• 10 Ozkaynak M, Wu DTY, Hannah K, Dayan PS, Mistry RD. Examining workflow in a pediatric emergency department to develop a clinical decision support for an Antimicrobial Stewardship Program. Appl Clin Inform 2018; 9 (02) 248-260
• 11 Evans RS, Olson JA, Stenehjem E. et al. Use of computer decision support in an antimicrobial stewardship program (ASP). Appl Clin Inform 2015; 6 (01) 120-135
• 12 Nichols KR, Petschke AL, Webber EC, Knoderer CA. Comparison of antibiotic dosing before and after implementation of an electronic order set. Appl Clin Inform 2019; 10 (02) 229-236
• 13 Dellit TH, Owens RC, McGowan Jr JE. Infectious Diseases Society of America, Society for Healthcare Epidemiology of America. et al; Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis 2007; 44 (02) 159-177
• 14 Ivers N, Jamtvedt G, Flottorp S. et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012; (06) CD000259
• 15 Lavoie CF, Plint AC, Clifford TJ, Gaboury I. “I never hear what happens, even if they die”: a survey of emergency physicians about outcome feedback. CJEM 2009; 11 (06) 523-528
• 16 Gerber JS, Prasad PA, Fiks AG. et al. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial. JAMA 2013; 309 (22) 2345-2352
• 17 Kramer J, Noronha S, Vergo J. A user-centered design approach to personalization - The key to successful design is grounding the choice of features and tools upon value to the end user. Commun ACM 2000; 43: 45-48
• 18 Constantine LL, Lockwood LA. Usage-centered software engineering: an agile approach to integrating users, user interfaces, and usability into software engineering practice. . Paper presented at: International Conference on Software Engineering. Portland, OR. Available at: https://ieeexplore.ieee.org/document/1201267. Accessed 2003
• 19 Harte R, Glynn L, Rodríguez-Molinero A. et al. A human-centered design methodology to enhance the usability, human factors, and user experience of connected health systems: A three-phase methodology. JMIR Human Factors 2017; 4 (01) e8
• 20 Lyon AR, Koerner K. User-centered design for psychosocial intervention development and implementation. Clin Psychol (New York) 2016; 23 (02) 180-200

Publication History

Received: 29 January 2020

Accepted: 16 September 2020

Article published online:
20 January 2021