Download PDF
Methods Inf Med 2014; 53(05): 389-405
DOI: 10.3414/ME13-02-0033
Focus Theme – Original Articles
Schattauer GmbH

Real-time Feedback on Nonverbal Clinical Communication

Theoretical Framework and Clinician Acceptance of Ambient Visual Design
A. L. Hartzler
1   The Information School, University of Washington, Seattle, Washington, USA
,
R. A. Patel
2   Department of Biomedical Informatics and Medical Education, Division of Biomedical and Health Informatics, University of Washington, Seattle, Washington, USA
,
M. Czerwinski
3   Microsoft Research, Redmond, Washington, USA
,
W. Pratt
1   The Information School, University of Washington, Seattle, Washington, USA
2   Department of Biomedical Informatics and Medical Education, Division of Biomedical and Health Informatics, University of Washington, Seattle, Washington, USA
,
A. Roseway
3   Microsoft Research, Redmond, Washington, USA
,
N. Chandrasekaran
3   Microsoft Research, Redmond, Washington, USA
,
A. Back
4   Department of Medicine, Division of Oncology, University of Washington, Seattle, Washington, USA
› Author Affiliations
Further Information

Publication History

received:21 September 2013

accepted:25 April 2014

Publication Date:
20 January 2018 (online)

   Buy Article Permissions and Reprints
Zoom Image

Summary

Introduction: This article is part of the Focus Theme of Methods of Information in Medicine on “Pervasive Intelligent Technologies for Health”.

Background: Effective nonverbal communication between patients and clinicians fosters both the delivery of empathic patient-centered care and positive patient outcomes. Although nonverbal skill training is a recognized need, few efforts to enhance patient-clinician communication provide visual feedback on nonverbal aspects of the clinical encounter.

Objectives: We describe a novel approach that uses social signal processing technology (SSP) to capture nonverbal cues in real time and to display ambient visual feedback on control and affiliation – two primary, yet distinct dimensions of interpersonal nonverbal communication. To examine the design and clinician acceptance of ambient visual feedback on nonverbal communication, we 1) formulated a model of relational communication to ground SSP and 2) conducted a formative user study using mixed methods to explore the design of visual feedback.

Methods: Based on a model of relational communication, we reviewed interpersonal communication research to map nonverbal cues to signals of affiliation and control evidenced in patient-clinician interaction. Corresponding with our formulation of this theoretical framework, we designed ambient real-time visualizations that reflect variations of affiliation and control. To explore clinicians’ acceptance of this visual feedback, we conducted a lab study using the Wizard-of-Oz technique to simulate system use with 16 healthcare professionals. We followed up with seven of those participants through interviews to iterate on the design with a revised visualization that addressed emergent design considerations.

Results: Ambient visual feedback on non-verbal communication provides a theoretically grounded and acceptable way to provide clinicians with awareness of their nonverbal communication style. We provide implications for the design of such visual feedback that encourages empathic patient-centered communication and include considerations of metaphor, color, size, position, and timing of feedback.

Conclusions: Ambient visual feedback from SSP holds promise as an acceptable means for facilitating empathic patient-centered nonverbal communication.

Keywords

Nonverbal communication - patient-provider communication - information display - social signal processing - formative evaluation - feedback - user-computer interface