User-Centered Redesign of a Clinical Decision Support System for Pneumonia in the Emergency Department

Abstract

Background

Deviation from evidence-based guidelines is common and associated with worse patient outcomes, especially in hectic emergency departments (EDs). Clinical decision support (CDS) systems can improve outcomes by promoting guideline adherence while allowing patient-specific adaptation. Implementation of a CDS system for pneumonia (“ePneumonia”) in the ED has been associated with improved guideline adherence and reduced 30-day mortality. However, adoption of ePneumonia has been hindered by a suboptimal user interface (UI).

Objective

This study aimed to redesign the ePneumonia UI to improve usability and adoption.

Methods

We conducted a user-centered design study involving ED clinicians at Vanderbilt University Medical Center. Across two rounds of one-on-one usability interviews with ED clinicians held via videoconference we (1) identified user requirements, and (2) iteratively refined a UI prototype. During each usability interview we presented realistic pneumonia cases, observed the clinician interact with a prototype, and elicited feedback with a semi-structured interview guide. We applied rapid thematic analysis and iteratively updated ePneumonia UI prototypes between interviews.

Results

Among 21 invited ED clinicians, 19 (90%) participated, including 15 attendings, 3 residents, and 1 advanced practice provider; 6 (32%) participants were women. Initial findings revealed that the original step-by-step UI did not align with the dynamic ED workflow. Clinicians expressed a need for both flexibility (e.g., skipping sections, overriding recommendations) and constraints (e.g., alerts for guideline deviations). We identified UI features to meet these needs that resulted in greater subjective usability including: a three-step navigation scheme, tiered information display, and multimodal indicators (text, icon, color) of agreement between clinician choices and CDS recommendations.

Conclusion

A user-centered design approach identified UI features that were associated with greater perceived usability of a CDS for pneumonia in the ED. Future work will evaluate real-world usability and adoption in a clinical trial.

Keywords

clinical decision support - user-centered design - electronic health record - pneumonia

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed and approved by the Vanderbilt University Medical Center Institutional Review Board. Informed consent was obtained from all participants.

Publication History

Received: 05 September 2025

Accepted: 09 February 2026

Article published online:
27 February 2026

• References

• 1 McGlynn EA, Asch SM, Adams J. et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348 (26) 2635-2645
• 2 Song Z, Kannan S, Gambrel RJ. et al. Physician practice pattern variations in common clinical scenarios within 5 US metropolitan areas. JAMA Health Forum 2022; 3 (01) e214698
• 3 Cotterill PG, Deb P, Shrank WH, Pines JM. Variation in chest pain emergency department admission rates and acute myocardial infarction and death within 30 days in the Medicare population. Acad Emerg Med 2015; 22 (08) 955-964
• 4 Trent SA, Jarou ZJ, Havranek EP, Ginde AA, Haukoos JS. Variation in emergency department adherence to treatment guidelines for inpatient pneumonia and sepsis: a retrospective cohort study. Acad Emerg Med 2019; 26 (08) 908-920
• 5 Coussens S, Ly DP. Variation in emergency department physician admitting practices and subsequent mortality. JAMA Intern Med 2025; 185 (02) 153-160
• 6 Chisholm CD, Weaver CS, Whenmouth L, Giles B. A task analysis of emergency physician activities in academic and community settings. Ann Emerg Med 2011; 58 (02) 117-122
• 7 Greenhalgh T, Howick J, Maskrey N. Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis?. BMJ 2014; 348: g3725
• 8 Bright TJ, Wong A, Dhurjati R. et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
• 9 Osheroff J. Improving Outcomes with Clinical Decision Support. HIMSS Publishing; 2012
• 10 Dutta S, McEvoy DS, Rubins DM, Dighe AS, Filbin MR, Rhee C. Clinical decision support improves blood culture collection before intravenous antibiotic administration in the emergency department. J Am Med Inform Assoc 2022; 29 (10) 1705-1714
• 11 Patterson BW, Pulia MS, Ravi S. et al. Scope and influence of electronic health record-integrated clinical decision support in the emergency department: a systematic review. Ann Emerg Med 2019; 74 (02) 285-296
• 12 Bennett P, Hardiker NR. The use of computerized clinical decision support systems in emergency care: a substantive review of the literature. J Am Med Inform Assoc 2017; 24 (03) 655-668
• 13 Chaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inform 2020; 11 (01) 46-58
• 14 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3 (01) 17
• 15 Jain S, Self WH, Wunderink RG. et al; CDC EPIC Study Team. Community-acquired pneumonia requiring hospitalization among U.S. adults. N Engl J Med 2015; 373 (05) 415-427
• 16 Ramirez JA, Wiemken TL, Peyrani P. et al; University of Louisville Pneumonia Study Group. Adults hospitalized with pneumonia in the United States: incidence, epidemiology, and mortality. Clin Infect Dis 2017; 65 (11) 1806-1812
• 17 Centers for Disease Control and Prevention. Underlying Cause of Death 1999–2015 on CDC WONDER Online Database, released December, 2016. Accessed October 23, 2017 at: http://wonder.cdc.gov/ucd-icd10.html
• 18 Dean NC, Jones JP, Aronsky D. et al. Hospital admission decision for patients with community-acquired pneumonia: variability among physicians in an emergency department. Ann Emerg Med 2012; 59 (01) 35-41
• 19 Jones BE, Brown KA, Jones MM. et al. Variation in empiric coverage versus detection of methicillin-resistant Staphylococcus aureus and Pseudomonas aeruginosa in hospitalizations for community-onset pneumonia across 128 US veterans affairs medical centers. Infect Control Hosp Epidemiol 2017; 38 (08) 937-944
• 20 Christensen MA, Nevers M, Ying J. et al. Simulated adoption of 2019 community-acquired pneumonia guidelines across 114 veterans affairs medical centers: estimated impact on culturing and antibiotic selection in hospitalized patients. Clin Infect Dis 2021; 72 (Suppl. 01) S59-S67
• 21 Mecham ID, Vines C, Dean NC. Community-acquired pneumonia management and outcomes in the era of health information technology. Respirology 2017; 22 (08) 1529-1535
• 22 Aronsky D, Haug PJ. Diagnosing community-acquired pneumonia with a Bayesian network. Proc AMIA Symp 1998: 632-636
• 23 Dean NC, Jones BE, Ferraro JP, Vines CG, Haug PJ. Performance and utilization of an emergency department electronic screening tool for pneumonia. JAMA Intern Med 2013; 173 (08) 699-701
• 24 Dean NC, Jones BE, Jones JP. et al. Impact of an electronic clinical decision support tool for emergency department patients with pneumonia. Ann Emerg Med 2015; 66 (05) 511-520
• 25 Dean NC, Vines CG, Rubin J. et al. Implementation of real-time electronic clinical decision support for emergency department patients with pneumonia across a healthcare system. AMIA Annu Symp Proc 2020; 2019: 353-362
• 26 Dean NC, Vines CG, Carr JR. et al. A pragmatic, stepped-wedge, cluster-controlled clinical trial of real-time pneumonia clinical decision support. Am J Respir Crit Care Med 2022; 205 (11) 1330-1336
• 27 Carr JR, Jones BE, Collingridge DS. et al. Deploying an electronic clinical decision support tool for diagnosis and treatment of pneumonia into rural and critical access hospitals: utilization, effect on processes of care, and clinician satisfaction. J Rural Health 2022; 38 (01) 262-269
• 28 Middleton B, Bloomrosen M, Dente MA. et al; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1, E1): e2-e8
• 29 Wright A, Sittig DF, Ash JS. et al. Lessons learned from implementing service-oriented clinical decision support at four sites: a qualitative study. Int J Med Inform 2015; 84 (11) 901-911
• 30 Chaparro JD, Beus JM, Dziorny AC. et al. Clinical decision support stewardship: best practices and techniques to monitor and improve interruptive alerts. Appl Clin Inform 2022; 13 (03) 560-568
• 31 Casey SD, Reed ME, LeMaster C. et al. Physicians' perceptions of clinical decision support to treat patients with heart failure in the ED. JAMA Netw Open 2023; 6 (11) e2344393
• 32 Miller K, Mosby D, Capan M. et al. Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support. J Am Med Inform Assoc 2018; 25 (05) 585-592
• 33 Carayon P, Wetterneck TB, Rivera-Rodriguez AJ. et al. Human factors systems approach to healthcare quality and patient safety. Appl Ergon 2014; 45 (01) 14-25
• 34 Lewis JR, Sauro J. Usability and user experience: design and evaluation. In: Salvendy G, Karwowski W. eds. Handbook of Human Factors and Ergonomics. 5th ed.. Wiley; 2021. . Accessed April 7, 2025 at: https://app.knovel.com/hotlink/khtml/id:kt012XG0G3/handbook-human-factors/what-is-usability
• 35 Weinger MB, Wiklund ME, Gardner-Bonneau DJ. eds. Handbook of Human Factors in Medical Device Design. CRC Press; 2010
• 36 Mandl KD, Kohane IS. No small change for the health information economy. N Engl J Med 2009; 360 (13) 1278-1281
• 37 Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016; 23 (05) 899-908
• 38 Ergonomics of human-system interaction Part 210: Human-centred design for interactive systems. Published online 2019. Accessed April 5, 2025 at: https://www.iso.org/standard/77520.html
• 39 Gosbee J, Gosbee LL. Usability evaluation in health care. In: Carayon P. ed. Handbook of Human Factors and Ergonomics in Health Care and Patient Safety. 2nd ed.. CRC Press; 2012: 543-553
• 40 Ward MJ, Chavis B, Banerjee R, Katz S, Anders S. User-centered design in pediatric acute care settings antimicrobial stewardship. Appl Clin Inform 2021; 12 (01) 34-40
• 41 Turer RW, Gradwohl S, Stassun J. et al. User-centered design and implementation of an interoperable FHIR application for pediatric pneumonia prognostication in a randomized trial. Appl Clin Inform 2024; 15 (03) 556-568
• 42 Jones BE, Collingridge DS, Vines CG. et al. CDS in a learning health care system: identifying physicians' reasons for rejection of best-practice recommendations in pneumonia through computerized clinical decision support. Appl Clin Inform 2019; 10 (01) 1-9
• 43 Anders S, Albert R, Miller A. et al. Evaluation of an integrated graphical display to promote acute change detection in ICU patients. Int J Med Inform 2012; 81 (12) 842-851
• 44 Anderson JW, Greenwood MR, Borsato GG, Kuttler KG. Alerting wisely: reducing inappropriate blood chemistry panel orders using a clinical decision support tool. J Healthc Qual 2020; 42 (01) 12-18
• 45 Ward MJ, Ferrand YB, Laker LF. et al. The nature and necessity of operational flexibility in the emergency department. Ann Emerg Med 2015; 65 (02) 156-161
• 46 Peltan ID, Bledsoe JR, Jacobs JR. et al. Effectiveness and safety of an emergency department code sepsis protocol: a pragmatic clinical trial. Ann Am Thorac Soc 2024; 21 (11) 1560-1571
• 47 Metlay JP, Waterer GW, Long AC. et al. Diagnosis and treatment of adults with community-acquired pneumonia. An official clinical practice guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med 2019; 200 (07) e45-e67
• 48 Figma. Accessed July 1, 2024 at: www.figma.com
• 49 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
• 50 Harris PA, Taylor R, Minor BL. et al; REDCap Consortium. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019; 95: 103208
• 51 Nielsen J. Jakob Nielsen's Ten Usability Heuristics. Nielsen Norman Group; 2020. . Accessed January 12, 2023 at: https://www.nngroup.com/articles/ten-usability-heuristics/
• 52 Holden RJ, Carayon P, Gurses AP. et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013; 56 (11) 1669-1686
• 53 Shneiderman Ben. Plaisant Catherine. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Pearson/Addison Wesley; 2004
• 54 Wiler JL, Welch S, Pines J, Schuur J, Jouriles N, Stone-Griffith S. Emergency department performance measures updates: proceedings of the 2014 Emergency Department Benchmarking Alliance Consensus Summit. Academic Emergency Medicine 2015; 22 (05) 542-553
• 55 Liu SW, Hamedani AG, Brown DFM, Asplin B, Camargo Jr CA. Established and novel initiatives to reduce crowding in emergency departments. West J Emerg Med 2013; 14 (02) 85-89
• 56 Salwei ME, Carayon P, Hoonakker PLT. et al. Workflow integration analysis of a human factors-based clinical decision support in the emergency department. Appl Ergon 2021; 97: 103498
• 57 Greenes RA, Bates DW, Kawamoto K, Middleton B, Osheroff J, Shahar Y. Clinical decision support models and frameworks: seeking to address research issues underlying implementation successes and failures. J Biomed Inform 2018; 78: 134-143
• 58 McCoy AB, Russo EM, Johnson KB. et al. Clinician collaboration to improve clinical decision support: the Clickbusters initiative. J Am Med Inform Assoc 2022; 29 (06) 1050-1059
• 59 Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record-based settings. JAMA Intern Med 2013; 173 (08) 702-704
• 60 Endsley MR, Cooke N, McNeese N, Bisantz A, Militello L, Roth E. Special issue on human-AI teaming and special issue on AI in healthcare. J Cogn Eng Decis Mak 2022; 16 (04) 179-181
• 61 Novak LL, Holden RJ, Anders SH, Hong JY, Karsh BT. Using a sociotechnical framework to understand adaptations in health IT implementation. Int J Med Inform 2013; 82 (12) e331-e344
• 62 Lyell D, Coiera E. Automation bias and verification complexity: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 423-431
• 63 Norman DA. The “problem” with automation: inappropriate feedback and interaction, not “over-automation.”. Philos Trans R Soc Lond B Biol Sci 1990; 327 (1241) 585-593
• 64 McDaniel RB, Burlison JD, Baker DK. et al. Alert dwell time: introduction of a measure to evaluate interruptive clinical decision support alerts. J Am Med Inform Assoc 2016; 23 (e1): e138-e141
• 65 Shneiderman B. The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings 1996 IEEE Symposium on Visual Languages. IEEE Comput. Soc. Press; 1996: 336-343