The Impact of Patient Interactive Systems on the Management of Pain in an Inpatient Hospital Setting: A Systematic Review

Keywords

pain management - medical informatics applications - patient participation - outcome assessment

Background and Significance

The management of acute pain in a hospital setting is a multifaceted, complex process made no less complicated by the fact that each patient's experiences with pain is subjective and unique. Providers are being taught to consider pain as the “sixth vital sign,” with this emphasis reinforced by the priority placed on patient feedback on how their pain was managed on the Hospital Consumer Assessment of Healthcare Providers and Systems survey, the most widely used metric for patient satisfaction.[1] Successful pain management goes beyond simply reacting to acute pain with medication, as evidenced by the Joint Commission, hospitals are required to involve patients in pain management treatment planning, to educating patients on pain management discharge plans, and to providing nonpharmacologic pain treatment modalities.[2]

Individuals experience pain differently, with demographic factors such as race/ethnicity, age, and gender[3] [4] and psychological factors such as anxiety and past experiences[5] [6] possibly impacting how patients perceive and respond to the treatment of pain. Complicating things further, patient satisfaction with pain control is not the same as patients simply endorsing the absence of pain. In fact, Pronovost and colleagues showed that patients were more likely to rate their overall satisfaction high if they perceived that their care providers did everything possible to control pain, regardless of how much pain the patients were actually in.[7]

In response to this need for individualized approaches, hospitals are turning to information technology to develop and implement systems to support an optimal multifaceted pain management process. Many of these applications have focused on automation of the pain assessment and documentation process,[8] [9] [10] or on the use of mobile applications to engage patients in tracking and managing their pain.[11] [12] Web-based intervention systems create opportunities to enhance patient education and engagement via interactivity,[13] [14] defined as the “extent to which users can participate in modifying the form and content of a mediated environment in real time.”[15] For the purposes of this review, we refer to inpatient interactive systems in a broad sense of patient/caregiver usage of technology that delivers one or more of the following features: health self-management, communication, education, or entertainment/distraction services in an inpatient hospital setting. These systems are designed to enhance or promote patient engagement through interacting with the system's components through specific electronic devices, inpatient portals, or inpatient television screens.

Patients who are more actively involved in their health care experience better health outcomes, lower health costs, and higher levels of satisfaction.[16] [17] [18] The use of patient interactive systems in the management of pain have been reported in the literature since the early 2000s when virtual reality (VR) systems were introduced to distract patients during painful therapy or treatment. In more recent years, the use of these systems has extended beyond a means of distraction to serve as a platform for patient–provider communication and enhance access to patient-standardized patient education. These systems can potentially allow for a more patient-centered care approach and an improved clinical workflow.

Previous systematic reviews explored patient engagement systems,[19] demonstrated the effects of patient interactive systems on patient engagement,[20] and their impact on patients' self-management of health, such as diabetes,[21] asthma,[22] weight loss,[23] and smoking cessation.[24] To our knowledge, no review has been conducted on the impact of patient interactive systems on pain management, particularly in an inpatient setting.

Objectives

We aimed to summarize the current state of scientific literature regarding the use of patient interactive systems designed for the management of pain within an inpatient hospital setting. Specifically, we sought to determine whether patient engagement through the use of interactive systems for pain management leads to improvements in clinical care, clinical workflows, patient outcomes, or user satisfaction.

Methods

Search Strategy

We systematically searched the literature to capture all publications relating to the impact of patient engagement and the use of interactive systems on clinical care, workflows, patient-reported outcomes, and user satisfaction. We registered the review protocol in PROSPERO,[25] and conducted searches across 8 databases: MEDLINE and Embase (both via Ovid), Cochrane Library (via Wiley), Web of Science, Scopus, Global Index Medicus, ClinicalTrials.gov, and WHO ICTRP. Additionally, we consulted the reference lists of relevant systematic reviews and hand-searched conference proceedings. In accordance with the Methodological Expectations for Cochrane Intervention Reviews guidelines,[26] we employed a combination of controlled vocabulary and natural language and we placed no limitations on date of publication or language. A complete search strategy is available in the [Supplementary Appendix] (available in the online version). Searches were first conducted in June 2018 and results were updated in May 2019. Results were compiled and de-duplicated using EndNote (Version X7).[27]

Studies that describe the implementation or use of a patient interactive system for the management of pain in an inpatient hospital setting were included in this review. Exclusion criteria included reports exclusively in outpatient or home care settings, not utilizing an interactive pain management system, and not engaged in pain management activities. Articles that did not report original data, such as narrative reviews or opinion pieces, were also excluded.

Screening and Study Selection

Three experienced researchers (health information professional, health informatics professional, and medical doctor) conducted the screening. Two independent screeners (R.A. and M.P.) reviewed each title and abstract for inclusion using Rayyan, a Web application that supports collaboration among researchers during screening and study selection.[28] Where discrepancies existed between the two screeners, a decision was reached through discussion or, where necessary, by a third screener (C.B.). Full-text screening followed title and abstract screening, again with the two independent screeners determining inclusion. Screeners recorded rationale for exclusion, which is reported in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram ([Fig. 1]).[29]

Fig. 1 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram.

Data Extraction and Classification

We developed data extraction forms that captured whether articles reported on specific outcome measures: (1) changes in clinical workflow; measuring a set of outcomes measures, which may include pre- and postcomparison studies; (2) patient engagement; measuring dimensions of patient participation, patient activation, patient engagement, patient motivation, or self-efficacy through patient reports, validated surveys such as the Patient Activation Measure, the Altarum Consumer Engagement Measure, or measures defined by authors such as “sense of presence”[30]; (3) user satisfaction (patient, parent, or nurse); measured through patient reports, validated surveys, or qualitative measures such as patient interviews; (4) changes in patient-reported pain level scores; measured through a validated clinical pain assessment tool; and (5) changes in electronic health record (EHR) pain documentation; measuring a set of outcomes measures, which include pre- and postcomparison studies.

Where available, we collected further information on the mechanism for gathering that data, such as the use of a validated survey or measurement tool. We also recorded the type of interactive tool being utilized, the number of participants, and whether it was an adult or pediatric population. One author developed the forms, which were then piloted and refined by the other two authors.

Risk of bias was assessed using ROBINS-I (Risk Of Bias In Non-Randomized Studies - of Interventions) for nonrandomized studies and GRADE (Grading of Recommendations Assessment, Development, and Evaluation) for randomized studies.[31] [32] Two independent screeners performed data extraction and risk of bias assessment using Qualtrics.[33] Discrepancies were resolved through discussion with a third screener. Due to the heterogeneity of the studies, meta-analysis was not possible.

Results

The combined search strategies identified 1,894 electronic records for the title and abstract screening phase, which yielded 58 potentially eligible studies for full-text screening. Of these, 18 met the study inclusion criteria and were included in the final qualitative synthesis phase ([Fig. 1]).

Ten studies explored the use of interactive pain management systems in an adult population, four in a pediatric population, and three included both an adult and pediatric population. There were 13 randomized controlled trials (RCTs) and 5 non-RCTs. All studies were published in English and three studies were conducted outside of the United States.

Study Quality

Results of the risk of bias assessment for RCTs are reported in [Fig. 2]. Only one trial was at low risk of bias for all quality criteria[34] while all others included some unclear or high risk of bias for some quality criteria. The risk of bias assessment for nonrandomized studies is reported in [Fig. 3]. Overall, the majority of studies were found to have an unclear risk of bias.

Fig. 2 The risk of bias assessment with Grading of Recommendations Assessment, Development, and Evaluation (GRADE) for randomized controlled trials. (A) Risk of bias summary. (B) Risk of bias graph.

Fig. 3 The risk of bias assessment with the Risk Of Bias In Non-Randomized Studies - of Interventions (ROBINS-I) tool for nonrandomized controlled trials. (A) Risk of bias summary. (B) Risk of bias graph.

Types of Pain Management Interactive Systems

There were two main types of pain interactive systems described: standalone systems (systems designed to include one specific function) and platform systems (systems designed to include more than one function). These are summarized in [Table 1] and described further below.

Table 1 Details of reviewed studies

Author	Category	Patient population	Brief description	Experimental type	Number of participants/sample	EHR integration
Tashjian et al, 2017	Entertainment/ Distraction system	Adult	Analyzed the impact of a one-time 3D VR intervention versus a 2D distraction video on patients suffering from acute abdominal pain with an average pain score of ≥3/10	Randomized	50 patients in each group (total = 100 patients)	No
Faber et al, 2013	Entertainment/ Distraction system	Pediatric & adult	Explored whether VR via distraction reduces pain during more than one wound care session per patient at regional burn unit in the Netherlands	Nonrandomized	36 patients	No
Li et al, 2011	Entertainment/ Distraction system	Adult	Explored effects of music therapy on pain reduction in breast cancer patients after radical mastectomy at a surgical unit in China. Control group patients chose preferred music and listened through a headphone connected to the MP3 player	Randomized	60 patients in each group (total = 120 patients)	No
Morris et al, 2010	Entertainment/ Distraction system	Adult	Examined feasibility and effect of a low-cost VR system, used with pharmacological analgesia, on reducing pain in adult burn patients undergoing physiotherapy treatment, compared with pharmacologic analgesia alone at a South African hospital	Randomized	11 patients	No
Carrougher et al, 2009	Entertainment/ Distraction system	Adult	Described the effects of adding VR to standard therapy in adult burn patients receiving active-assisted ROM physical therapy, by assessing pain scores before and after therapy on two consecutive days	Randomized	39 patients	No
Konstantatos et al, 2009	Entertainment/ Distraction system	Adult	Examined whether preprocedural VR-guided relaxation added to PCA with morphine reduced pain severity during dressings changes in burns patients	Randomized	43 patients in each group (total = 86 patients)	No
Sharar et al, 2007	Entertainment/ Distraction system	Pediatric & adult	Examined efficacy and side effects of VR distraction analgesia and patient factors associated with VR analgesic efficacy in burn patients who require ROM physical therapy. Study participants being pooled from 3 other unpublished studies	Randomized	88 patients	No
Das et al, 2005	Entertainment/ Distraction system	Pediatric	Investigated whether VR, decreases procedural pain with acute burn injuries in which patients acted as their own controls though a series of 11 trials	Randomized	7 patients	No
Hoffman et al, 2001	Entertainment/ Distraction system	Pediatric & adult	Compared the efficacy of VR with the efficacy of a conventional treatment during at least three separate therapy sessions with multiple burn patients	Randomized	7 patients	No
Hoffman et al, 2000	Entertainment/ Distraction system	Adult	Explored use of VR to distract burn patients from pain during physical therapy treatment. Each patient spent 3 min of physical therapy with no VR and 3 min of physical therapy with VR	Randomized	12 patients	No
Martorella et al, 2012	Communication/ Educational system	Adult	Investigated the preliminary effects of a virtual nursing Web-based intervention to improve pain relief in patients undergoing cardiac surgery. The intervention includes a preoperative 30-min Web-based session and 2 brief face-to-face postoperative sessions. It generates reflective activities and specific personalized educational messages transmitted through videos of a virtual nurse, animations, stories, and texts	Randomized	30 patients in each group (total = 60 patients)	No
Martorella et al, 2013	Communication/ Educational system	Adult	Reported the development, validation, feasibility and acceptability of a virtual nursing Web-based intervention for postoperative pain self-management in adults scheduled for cardiac surgery. The intervention includes a preoperative 30-min Web-based session and 2 brief face-to-face postoperative sessions. It generates reflective activities and specific personalized educational messages transmitted through videos of a virtual nurse, animations, stories, and texts	Randomized	30 patients in each group (total = 60 patients)	No
McDonall et al, 2019	Communication/ Educational system	Adult	Evaluated the efficacy of a bedside multimedia intervention accessed through iPads, designed to engage patients in their recovery process after total knee replacement surgery. The intervention contains tailored educational information based on each day of recovery and specific patient recovery goals, which are facilitated by clinicians	Randomized	104 patients in the intervention group (total = 241)	No
Wirz et al, 2017	Personalized PCA device	Adult	Evaluated the safety, efficacy, and usability of a novel bedside PCA dispenser. The device has three main features: a drug dispensing unit, a radiofrequency identification wristband used for patient's registration, and a PillBox, which is a patient-specific, mouth-actuated, disposable receptacle, from which the patient receives the pill	Randomized	27 patients in the test group (total = 70 patients)	No
Aldekhyyel et al, 2018	Interactive platform system	Pediatric	Described the implementation of a pain management interactive tool, which allows for real-time patient-reported pain assessments through inpatient television screens, integrated with the hospital's EHR, medication dispensing machine and the nursing phones. The study measured the effects of implementation by extracting pre- and postimplementation pain assessment data (22 mo) from the EHR	Nonrandomized	56,931 patient records (2,447 unique)	Yes
Aldekhyyel et al, 2018	Interactive platform system	Pediatric	Described end-users' (nurses and parents) perspectives on using a pain management interactive tool to report pain using a mixed-methods approach. The tool integrated 4 standalone technologies (EHR, medication dispensing machine, nurse phones, and inpatient TV screens)	Nonrandomized	30 parents and 59 nurses (total = 89 users)	Yes
Patmon et al, 2016	Interactive platform system	Adult	Described the perceptions of nurses who use an interactive tool in their daily work. The tool is built for patient–provider communication, patient access to the EHR, patienteducation, and distraction therapy. It is accessed through iPads in the outpatient clinics and television screens in the inpatient rooms	Nonrandomized	38 nurses	No
Rao-Gupta et al, 2018	Interactive platform system	Pediatric	Described a quality improvement project to develop new workflows to integrate an interactive patient care technology system (designed for the management of pain accessed through the inpatient television screens) with the hospital's automated medication dispensing system and integrate the system with the hospital's EHR	Nonrandomized	NA	Yes

Abbreviations: 2D, two-dimensional; 3D, three-dimensional; EHR, electronic health record; PCA, patient-controlled analgesia; ROM, range of motion; VR, virtual reality.

Standalone Interactive Systems

There were 14 stand-alone interactive systems described, of which 9 focused on the use of VR designed to distract patients from abdominal pain,[35] or burn patients from pain during physical therapy sessions[30] [34] [36] [37] [38] or wound/dressing changes.[39] [40] [41] One system described using interactive music therapy,[42] two other described a Web-based virtual nurse designed for patient–provider communication and patient education,[42] [43] and one described a bedside educational multimedia program intervention accessed through iPads.[44] Finally, one paper described a personalized patient-controlled analgesia (PCA) device for oral medications.[45]

VR Distraction Systems

The use of VR systems in the management of pain offers a psychologically based approach to help minimize pain. These systems are computer-generated environments, where “…patients typically use a head-mounted three-dimensional visual screen to interact with a computer environment to draw attention away from their pain.”[39] Carrougher et al described why VR is helpful in reducing patient pain by explaining that “…individuals have a limited amount of attention that can be divided between incoming stimuli, and pain, which requires attention. Because VR is a highly attention-grabbing experience, it can be an effective psychological pain control technique.”[36]

Hoffman et al published the earliest studies (conducted in 2000 and 2001) that examined the use of VR by burn patients during physical therapy sessions.[30] [38] Studies that followed examined the efficacy of using VR in controlling pain in children with acute burn injuries,[41] factors that influenced the efficacy of VR in distracting patients during physical therapy,[36] [37] and examined whether preprocedural VR-guided relaxation added to morphine-reduced pain severity during dressings changes in burn patients.[40] Later studies focused on testing the feasibility and potential effects of a low-cost VR systems in reducing patient pain during physiotherapy in a developing country,[34] examined the repeated use of VR to control pain during wound dressing changes,[39] and studied the impact of VR on patients suffering from acute abdominal pain.[35]

Interactive Music

Among the reviewed studies that examined the use of entertainment/distraction in the management of pain during therapy, Li et al was the only study that described the use of an interactive music therapy intervention.[42] The intervention was designed to allow patients to choose their preferred music, control music volume, and listen through a personal headphone connected to an MP3 player. The authors reported the effects of the intervention on reducing pain after radical mastectomy in cancer patients in China.

Communication/Educational Systems

There were three studies that examined and evaluated the development of an educational/communication system, which is designed to deliver personalized pain-relieving educational videos through different types of output devices. Two studies investigated the development of a virtual nursing Web-based intervention to improve pain relief in cardiac surgery patients and reported on the preliminary effects of the system.[42] [44] The Web application generates reflective personalized activities and specific educational messages designed for patients based on their beliefs and attitudes. The system messages are transmitted through videos of a virtual nurse, animations, stories, and texts. The third study evaluated the efficacy of a bedside multimedia intervention accessed through iPads. The intervention is designed to engage patients in their recovery process after total knee replacement surgery.[44]

Personalized PCA Device

One study described a novel oral PCA dispenser providing on-demand pain medication at the bedside. The device has three main features: a drug dispensing unit, a radiofrequency identification wristband used for patient's registration, and a PillBox; “a patient-specific, mouth-actuated, disposable receptacle from which the patient receives the pill.”[45]

Interactive Platform Systems

Interactive platform systems are systems which include interactive prompts meant to better engage patients in their care during a hospital encounter and improve their patient experience. These systems display standardized patient education material, care management, patient–provider communication, and/or entertainment features, which are accessed through patient-facing interfaces such as inpatient bedside television screens or portable devices. Other systems' capabilities may include EHR integration and real-time patient–provider communication and alerts.[46]

There were four studies describing the use of interactive platform systems built to support patient engagement and timely nursing pain reassessment documentation practices. Two described the effects of integrating an interactive platform system, accessed through the inpatient television screens, with the hospital's EHR system, medication dispensing machine, and the nursing staff call system.[47] [48] The third study described a similar interactive platform system for bedside pain reporting through the television; however, the system was not integrated with the nursing staff call system.[49] The fourth study described the perceptions of nurses who use an interactive system via iPads and televisions in their daily work.[50] This system was mainly built to support patient–provider communication and patient access to the EHR.

Outcome Measures

We extracted 28 outcome measures from the 18 papers ranging between one and four outcomes per study. These outcomes mapped to five themes: (1) changes in patient-reported pain levels; (2) patient engagement; (3) user satisfaction; (4) changes in clinical workflow; and (5) changes in clinical documentation practices.

Standalone Interactive Systems

VR and Interactive Music Distraction Systems

Studies that described the use of VR and interactive music as an entertainment/distraction intervention for the management of pain all reported changes in patient-reported pain levels, as a main outcome measure, using a Visual, Graphical, Numeric, or a modified Faces Pain Rating Scale. A total of 8 out of 9 VR distraction pain management intervention studies reported a statistically significant decrease in patient-reported pain level scores ([Table 2]).[30] [34] [35] [36] [37] [38] [39] [41] Konstantatos et al was the only study that reported that use of VR-guided relaxation during dressings changes resulted in higher pain scores when compared with the use of morphine alone (p = 0.003) (95% confidence interval: 0.6–2.8).[40]

Table 2 Reported outcome measures from reviewed studies

Author	Category	Outcome measures
		Main outcome		Measurement	Main findings
Tashjian et al, 2017	Entertainment/ Distraction system	1-	Changes in patient-reported pain levels	Pre- and postintervention pain scores	Mean pain reduction in VR cohort was greater than in controls (–1.3 vs. –0.6 points; p = 0.008). In GI subgroups, mean pain reduction in VR cohort (pre = 5.72, post = 4.18; p = 0.016). Multivariable regression analysis adjusting for age, race, ethnicity, and gender, VR was a predictor of pain reduction independent of pain origin (β coefficient = –0.65 points; p = 0.05)
Faber et al, 2013	Entertainment/ Distraction system	1-	Changes in patient-reported pain levels	Visual Analog Thermometer (10 cm tall burn-specific pain rating device)	Pain ratings during wound debridement were statistically lower when patients were in VR on days 1, 2, and 3, and although not significant beyond day 3, the pattern of results from days 4, 5, and 6 were consistent with the notion that VR continues to reduce pain when used repeatedly
Li et al, 2011	Entertainment/ Distraction system	1-	Changes in patient-reported pain levels	General Questionnaire and Chinese version of Short-Form of McGill Pain Questionnaire. Visual Analogue Scale, and Present Pain Intensity scores	Pain scores measured at baseline and 3 posttests. Music therapy reduced Pain Rating Index (PRI-total) score in intervention group compared with control group with a mean difference of –2.38, –2.41, and –1.87 for the 1st, 2nd, and 3rd posttests, respectively
Morris et al, 2010	Entertainment/ Distraction system	1-	Changes in patient-reported pain levels	Numeric Pain Rating Scale	Patients reported a marginal (p = 0.06) to (p = 0.13) difference between the two sessions (analgesia with VR and analgesia without VR) in reducing pain
Carrougher et al, 2009	Entertainment/ Distraction system	1-	Patient engagement	0–100 mm Graphic Rating Scale	VR object “presence” was measured. Approximately half of the patients (51.3%) rated their level of presence at greater than 35 mm
		2-	Changes in patient-reported pain levels	0–100 mm Graphic Rating Scale	VR reduced all Graphic Rating Scale pain scores (worst pain, time spent thinking about the pain, and pain unpleasantness by 27, 37, and 31%, respectively), relative to the no VR condition
Konstantatos et al, 2009	Entertainment/ Distraction system	1-	Changes in patient-reported pain levels	10 cm VisualAnalogue Rating Scale	The group receiving VR relaxation plus morphine PCA reported higher pain intensities during the dressing change (mean = 7.3) compared with patients receiving morphine PCA alone (mean = 5.3) (p = 0.003) (95% CI: 0.6–2.8)
Sharar et al, 2007	Entertainment/ Distraction system	1-	Patient engagement	Sense of “presence” (0–100 Graphical Analog Scale)	Children reported higher subjective reports of “presence” using VR and “realness” of VR than adults. Age did not affect analgesic effects of VR distraction
		2-	Changes in patient-reported pain levels	0–100 mm Graphical Rating Scale	Compared with standard analgesic treatment alone, addition of VR resulted in pain ratings reduction for worst pain intensity (20% reduction), pain unpleasantness (26% reduction), and time spent thinking about pain (37% reduction)
Das et al, 2005	Entertainment/ Distraction system	1-	User satisfaction	Interviews with nurses, parents and children	General positive feedback regarding the effects of VR in distraction, which had an influence in reducing sensitivity to pain
		2-	Changes in patient-reported pain levels	Modified Faces Pain Scale	Average pain scores for pharmacological analgesia only was 4, compared with VR with pharmacological analgesia (average pain score was 1.3)
Hoffman et al, 2000	Entertainment/ Distraction system	1-	Patient engagement	Sense of “presence” (0–100 Visual Analog Scale)	Patients reported mean presence in VR was 63.67 mm, and mean realism of virtual objects was 51.92 mm
		2-	Changes in patient-reported pain levels	0–100 mm Visual Analog Scale	Patients completed 5 subjective pain ratings. They reported less pain, when distracted with VR, magnitude of pain reduction by VR was statistically significant
Hoffman et al, 2001	Entertainment/ Distraction system	1-	Patient engagement	Sense of “presence” (0–100 Visual Analog Scale)	Patients experiencing VR were asked after each physical therapy session. All, except one patient, reported a score above 50
		2-	Changes in patient-reported pain levels	0–100 mm Visual Analog Scale	Patients completed 5 subjective pain ratings. Pain ratings were statistically lower with VR. Repeated use of VR resulted in magnitude of pain reduction
Martorella et al, 2012	Communication/ Educational system	1-	Changes in patient-reported pain levels	Barriers Questionnaire-II, Pain Catastrophizing Scale	Experimental group reported less pain interference when breathing/coughing (p = 0.04). Day 7 postsurgery, experimental group reported fewer pain-related barriers: (mean: 10.6) vs. controls (mean 15.8) (p = 0.02). Both groups mean revealed lower tendency to catastrophize pain presurgery
Martorella et al, 2013	Communication/ Educational system	1-	User satisfaction	Questionnaires of acceptability with 10 multiple-choice questions	Most participants indicated that the strategies proposed responded to their needs (96%) and that information provided helped control pain and lessen worries (93%)
McDonall et al, 2019	Communication/ Educational system	1-	Changes in clinical workflow	Length of hospital stay (LOS)	Significant reduction in LOS for the intervention group 5.29 d vs. control group 6.29 d (p = 0.04)
		2-	Patient engagement	Patient activation measure	No significant positive difference was reported between both groups. On day 3 after surgery, there was a reduction in the proportion of patients at level 3 and 4 activation in both groups from baseline scores (prior to admission). Control group significantly higher (20% decrease)
		3-	User satisfaction	Global satisfaction questions Net Promoter Score (recommend the health service to a family or friend)	Intervention group reported significantly higher mean satisfaction with acute care experience 9.26 versus 8.58 control group (p = 0.01) and higher NET promoter score 9.27 versus 8.67 control group (p = 0.02)
		4-	Changes in patient-reported pain levels	Pain intensity (Numerical Rating Scale 0–10), Pain quality (American Pain Society Outcome Questionnaire–Revised version), and Pain treatment– analgesic management (medical record audit)	Postoperative day 3: Pain intensity: significant reduction (1.01) in mean pain scores between groups. 64% of controls had a day 3 score ≥7, compared with 44% of intervention group (p = 0.04). Pain quality: interventions perceived they received more pain relief in previous 24 h (mean = 7.67 vs. mean = 7.07 (p = 0.05). Significantly more intervention patients reported using deep breathing to relieve pain (82.2% vs. 68%) (p = 0.02). Pain treatment: intervention group received significantly higher amounts of pro re nata oxycodone (daily amount (mg) – mean = 16.1 vs. control group 10.8, p = 0.001)
Wirz et al, 2017	Personalized PCA device	1-	Changes in clinical workflow	Value-stream mapping (comparison between pre- and postimplementation)	Medication provision process preimplementation comprised of 8 steps. Postimplementation was 3 steps
		2-	Patient engagement	“Efficacy measure”–data recorded by the device for each patient and questionnaires filled out by patients	Success rate of 90% for pill intake upon patient's request
		3-	User satisfaction	“Usability measure” measured using questionnaires filled out by patients and medical staff	At least 80% of patients and medical staff were satisfiedwith device use and recommend its use
		4-	Changes in patient-reported pain levels	0–10 Numeric Rating Scale	Patients reported pain levels before and after pill intake using PCA. They reported significantly less pain, both at rest and in movement, from first postoperative day
Aldekhyyel et al, 2018	Interactive Platform System	1-	Changes in documentation	% pain documentation occurrences (comparison between pre- and postimplementation)	Modest increase in mean timely documentation rates based on nursing documentation standards (26.1% vs. 32.8%, a % increase of 25.7%; p < 0.001) along with decreased median time to pain reassessment documentation (29 min vs. 25 min, % decrease of 13.8%; p < 0.001)
		2-	Changes in clinical workflow	Value-stream mapping (comparison between pre- and postimplementation)	Pain management tool interfaced 4 standalone technologies. The workflow is triggered when pain medications are dispensed by sending an automatic pain assessment rating question via the patient's television at a predefined time
		3-	Patient engagement	Usage rates based on responses to system prompts	Usage rates were low with 6.5% for the level of pain prompt and 13.3% for the other nonpharmacologic strategies to help with pain prompt
Aldekhyyel et al, 2018	Interactive Platform System	1- User satisfaction		2 survey instruments: closed-ended (5 point Likert scale) and open-ended questions	Parents were more satisfied (90%) compared with nurses (50%). Timely reassessments of pain were the most valuable feature reported. Qualitative analysis of nurses' responses yielded 6 themes for system benefits and 12 for challenges
Patmon et al, 2016	Interactive Platform System	1-	User satisfaction	Interviews using a focused rapid ethnographic evaluation	Participants reported findings of using the system effective for distraction, great functionality for patients and nurses, has implications for clinical practice, and needs additional training to improve usage
Rao-Gupta et al, 2018	Interactive Platform System	1-	Changes in clinical workflow	Value-stream mapping (comparison between pre- and postimplementation)	New workflow included 2 main improvements: admission assessment questions for healthcare team and integrations between interactive tool with EHR and medication dispensing system
		2-	User satisfaction	Proportion of “always” responses to the (Child HCAHPS, 2014–2016) pain question	Proportion of family satisfaction responses to the question ‘‘Did staff do everything they could to manage your child's pain?’' increased from year 2014 to year 2016 (p = 0.006)
		3-	Changes in documentation	# documentation occurrences	Documentation of nonpharmacologic interventions: year 2014 (# unique patients = 2,462), year 2015 (# unique patients = 2,684), year 2016 (# unique patients = 2,970)

Abbreviations: CI, confidence interval; EHR, electronic health record; GI, gastrointestinal; PCA, patient-controlled analgesia; VR, virtual reality.

In addition to reporting changes in patient-reported pain levels as a main outcome measure, four VR intervention studies reported levels of patient engagement and one reported levels of patient satisfaction as a secondary outcome measure. Studies that reported levels of patient engagement[30] [36] [37] [38] used a 1 to 100 Visual or Graphical Analog scale to measure sense of presence “illusion of being inside the computer-generated environment”[30] and engagement while using the VR system.[30] Hoffman et al reported average engagement mean scores in both of their studies.[30] [38] Carrougher et al showed that approximately half of the adult burn patients rated their level of presence and engagement with the VR during physical therapy treatment, at lower average scores,[36] while Sharar et al showed that children reported higher levels of engagement than adults.[37]

Among this group, only one study reported levels of satisfaction with the use of VR as a secondary outcome. The authors measured levels of satisfaction through a series of interviews with parents and nurses. Nursing staff indicated that the use of VR during wound dressing changes was helpful in distracting children from pain and “all parents agreed with the positive effects of VR in pain management for their child. They all commented that the child's anxiety level was perceptibly less when using VR, and the child looked forward to playing the VR game.”[41]

The interactive music intervention study measured pain scores at baseline and three posttests reporting on the Pain Rating Index (PRI-total) score from baseline. Music therapy was found to reduce the PRI-total score in the intervention group compared with the control group.[42]

Communication/Educational Systems

The two studies published by Martorella et al describing the use of a virtual nursing Web-based intervention for self-management of pain postcardiac surgery, reported changes in patient pain levels using the Barriers Questionnaire-II.[43] Patients in the intervention group reported fewer pain-related barriers (mean: 10.6) than patients in the control group (mean: 15.8, p = 0.02). In their follow-up study, the authors reported the acceptability of the virtual nursing Web-based intervention, by measuring the perceptions of patients using a questionnaire.[51] Most of the patients indicated that the strategies proposed responded to their needs and that the information provided helped control pain and lessen worries.

A recent study by McDonall et al evaluated the efficacy of a clinician-facilitated, bedside multimedia intervention for patients recovering from total knee replacement surgery.[44] The study described changes in pain-reported outcomes postoperative day 3 as a main outcome measure using three different measures: pain intensity (Numerical Rating Scale), pain quality (American Pain Society Outcome Questionnaire–Revised), and pain treatment analgesic management (medical record audit). Patients in the intervention group reported a lower mean worst pain score (6.05) than the control group (7.05, p = 0.04). Secondary outcomes included patient activation, length of hospital stay, knee function, and satisfaction with care. The intervention group had lower length of stay (5.29 days vs. 6.29 days, p = 0.04), higher levels of patient activation (45.1% level 4 activation vs. 27.1%, p = 0.04), and higher mean satisfaction with acute care experience (9.26 vs. 8.58, p = 0.01).

Personalized PCA Device

The study of the oral PCA dispenser reported an improvement in the clinical workflow from a total of eight to three steps. The authors also reported three secondary outcome measures: (1) high level of patient engagement (success rate for using the device of 90%); (2) lower patient-reported pain levels during day 1 and day 2 postoperative during rest (33.56% reduction, p = 0.0058) and movement (28% reduction, p = 0.0012); and (3) high satisfaction with the usability of the device as indicated by both patients and medical staff (80% were satisfied).[45]

Interactive Platform Systems

Two studies describing the implementation an interactive pain management platform system accessed through inpatient television screens, reported improvements in patient pain reassessment nursing documentation practices by calculating the percent of pain documentation occurrences pre- and postsystem implementation. The authors also reported: (1) improvement in clinical workflow by implementing nursing automatic alerts to conduct timely reassessments of pain, and (2) low levels of patient engagement with the system by calculating system usage rates.[47] In their follow-up study, the authors captured the perceptions of nurses and parents with the use of the system. Parents were more satisfied with the experience compared with nurses and both nurses and parents indicated that timely reassessments of pain was the most valuable system feature.[48]

A second group of authors described a quality improvement project to integrate a similar pain management interactive system with the hospital's EHR and medication dispensing system. The authors reported an increase in patients' satisfaction scores with the hospital's pain management initiatives, as a result of the integration. The authors also reported an increase in documentation of nonpharmacologic interventions.[49]

Patmon et al interviewed nurses to capture their perceptions with using an interactive patient engagement technology during their daily clinical practice. Nurses reported effectiveness of the tool for distraction therapy, satisfaction with the functionality of the tool, positive implications for clinical practice, and the need for additional training to improve usage.[50]

Discussion

Our systematic review identified the types of patient interactive systems used in the inpatient setting for the management of pain and the impact of these systems on controlling patient pain levels, hospital workflows, patient engagement, and user satisfaction. The majority of the studies included in our review were VR systems used for distracting burn patients during physical therapy treatment or wound dressing changes. These types of studies mainly reported the effects of using VR systems on controlling patient pain levels during hospitalization. These systems along with the communication/educational systems, which were included in our review, have shown to be effective in managing pain levels for patients.

With recently published studies, we found that there is a smaller but growing body of evidence describing the use of interactive platform systems promoting self-management of pain, increasingly with integration into different hospital technology systems. These interactive systems earned positive feedback from users, increased levels of satisfaction, and resulted in improved clinical workflows. Our review also revealed that interactive platform systems designed to support more than one aspect of pain management during hospitalization can potentially align with national pain management standards. Specifically, these systems may assist with increasing patient engagement in pain management treatment planning through education, providing nonpharmacologic pain treatment modalities, and facilitating reassessment and timely responses to patient's pain through automatic documentation of response(s) to pain interventions.[2] Additionally, some studies demonstrated the feasibility of integrating these systems with the EHR to support clinical documentation, create standardized fields for patient-generated pain assessment data, and allow patients to access their records. Having the ability to integrate these systems with the EHR system, taking into account a standardized pain information model,[52] [53] may potentially be of great value for health care organizations seeking the adoption of these interactive pain management systems, support secondary use for research purposes,[54] and create an opportunity to study the effects of these systems on reducing patient pain levels.

While the different interactive pain management platform systems included in our review have been well received by users, the number of patients/parents and nurses that used these systems was relatively low. To determine areas of improvement and enhance the systems' use among both patients/parents and clinicians/nurses, it is essential to conduct a usability evaluation study.[55] [56] [57] Expanding the use of these interactive platform systems for the management of other conditions may have the potential to support many patient engagement initiatives.[58] [59] [60] [61] [62]

Limitations of this review include the different definitions of interactivity, patient engagement, and outcome measures, making direct comparisons difficult. The differences in definitions, populations, and study designs led to heterogeneity, which in turn made a meta-analysis infeasible. This lack of a meta-analysis limits our ability to draw conclusions regarding the efficiency, efficacy, and effectiveness of one type of pain interactive system when compared with a different type of system. This is especially true when examining entertainment/distraction systems versus interactive platform systems, due to lack of research studies that report the effects of interactive platform systems in managing patient pain levels.

Another limitation is related to the sample sizes of some of our reviewed studies. Four of the nine reviewed VR studies had a very small sample size ranging from 7 to 12 patients. The reduced sample size limits the ability to generalize the results of these studies, and further research with a larger sample size may provide better understanding of the usefulness and effectiveness of those specific VR systems as a nonpharmacological approach to pain relief.

The impossibility of blinding participants and assessors in these studies also introduces the possibility of bias. Some studies lacked the specific details related to addressing the risk of bias, while other studies entirely did not mention it. This in turn resulted in an unclear risk of bias in the majority of the studies included in our review.

Finally, the five outcome measure themes described in our review were not equally represented in the literature and were heterogeneous within outcome measure themes. Limiting our review to those articles describing only pain management interactive patient care systems and its application in the inpatient care setting likely influenced the representation of outcome measures we observed. By restricting our review, we may have also unintentionally removed meaningful details from other patient interactive systems.

We also observed a diversity in evaluation criteria and a lack of a standardized framework for measuring the efficiency and effectiveness of interactive pain management interactive systems. Improved standardized evaluation assessments will help to improve our ability broadly to assess and improve health outcomes in existing and future pain management interactive systems.

Conclusion

The use of inpatient interactive systems in the management of pain is an emerging area of interest for researchers and health care providers in the era of modern health care technology and increased focus on patient engagement. These systems have primarily been entertainment-focused and built to distract patients during a treatment procedure and have shown an impact in lowering levels of patient-reported pain scores. Inpatient interactive pain management systems integrated with the hospital's EHR further facilitate patient–provider communication, patient education, and self-reporting; they show promising effects on timely pain assessments, increasing patient satisfaction, and patient engagement. Further high-quality studies with a more standardized approach to implementation and assessment are necessary to reinforce and validate these findings.

Clinical Relevance Statement

Addressing a hospitalized patient's pain through the use of other nonpharmacological measures has been the focus of many national standards and is an essential part of clinical care. Patient interactive systems continue to evolve in support of a more patient-centric approach to the management of pain. Findings of this study contribute to research efforts that evaluate the use of health information technology systems and their impact on patient care.

Multiple Choice Questions

1. Which of the following can be considered one of the benefits of integrating patient interactive systems with the hospital's EHR system?

   • Increasing security and privacy measures.

   • Enhancing the quality of data being stored.

   • Collecting and reporting quality indicators.

   • Maintaining active patients list.

   Correct Answer: The correct answer is option b. Inpatient interactive systems can be designed to automatically collect and store patient-generated reported outcomes. Having these systems integrated with the EHR has the potential to meet regulatory compliance standards and ultimately enhance the quality of data being stored in the EHRs for secondary quality use.

2. Which of the following is defined as the “extent to which users can participate in modifying the form and content of a mediated environment in real time”?

   • Interoperability.

   • Integration.

   • Interactivity.

   • Incorporation.

   Correct Answer: The correct answer is option C. Steuer was considered one of the early scholars that defined interactivity by focusing on the process of real-time participation between the mediated environment and the users.

• References

• 1 Hospital HCAHPS. Published 2017. Available at: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-instruments/hospitalqualityinits/hospitalHCAHPS.html . Accessed February 1, 2019
• 2 The Joint Commission. R 3 Report | Requirement, Rationale, Reference; 2017 . Available at: www.jointcommission.org . Accessed February 1, 2019
• 3 Wandner LD, Scipio CD, Hirsh AT, Torres CA, Robinson ME. The perception of pain in others: how gender, race, and age influence pain expectations. J Pain 2012; 13 (03) 220-227
• 4 Anderson KO, Mendoza TR, Valero V. , et al. Minority cancer patients and their providers: pain management attitudes and practice. Cancer 2000; 88 (08) 1929-1938
• 5 Rhudy JL, Meagher MW. Fear and anxiety: divergent effects on human pain thresholds. Pain 2000; 84 (01) 65-75
• 6 Linton SJ, Shaw WS. Impact of psychological factors in the experience of pain. Phys Ther 2011; 91 (05) 700-711
• 7 Hanna MN, González-Fernández M, Barrett AD, Williams KA, Pronovost P. Does patient perception of pain control affect patient satisfaction across surgical units in a tertiary teaching hospital?. Am J Med Qual 2012; 27 (05) 411-416
• 8 de Rond ME, de Wit R, van Dam FS, Muller MJ. A pain monitoring program for nurses: effects on communication, assessment and documentation of patients' pain. J Pain Symptom Manage 2000; 20 (06) 424-439
• 9 Samuels JG, Kritter D. Pain management documentation. Comput Informatics Nurs 2011; 29 (09) 512-518
• 10 Sikka K, Ahmed AA, Diaz D. , et al. Automated assessment of children's postoperative pain using computer vision. Pediatrics 2015; 136 (01) e124-e131
• 11 Symer MM, Abelson JS, Milsom J, McClure B, Yeo HL. A mobile health application to track patients after gastrointestinal surgery: results from a pilot study. J Gastrointest Surg 2017; 21 (09) 1500-1505
• 12 Agboola S, Kamdar M, Flanagan C. , et al. Pain management in cancer patients using a mobile app: study design of a randomized controlled trial. JMIR Res Protoc 2014; 3 (04) e76
• 13 Neary PM, Sung R, Corrigan M, O'Donovan M, Cahill RA, Redmond HP. The benefits of an interactive, individualized online patient pathway for patients undergoing minimally invasive radioguided parathyroidectomy: a prospective, double-blinded, randomized clinical trial. Surg Innov 2010; 17 (03) 236-241
• 14 Riva S, Camerini A-L, Allam A, Schulz PJ. Interactive sections of an Internet-based intervention increase empowerment of chronic back pain patients: randomized controlled trial. J Med Internet Res 2014; 16 (08) e180
• 15 Steuer J. Defining virtual reality: dimensions determining telepresence. J Commun 1992; 42 (04) 73-93
• 16 Hibbard JH, Greene J. What the evidence shows about patient activation: better health outcomes and care experiences; fewer data on costs. Health Aff (Millwood) 2013; 32 (02) 207-214
• 17 Fisher B, Britten N. Patient access to records: expectations of hospital doctors and experiences of cancer patients. Br J Gen Pract 1993; 43 (367) 52-56
• 18 Maly RC, Bourque LB, Engelhardt RF. A randomized controlled trial of facilitating information giving to patients with chronic medical conditions: effects on outcomes of care. J Fam Pract 1999; 48 (05) 356-363
• 19 Prey JE, Woollen J, Wilcox L. , et al. Patient engagement in the inpatient setting: a systematic review. J Am Med Inform Assoc 2014; 21 (04) 742-750
• 20 Tang C, Lorenzi N, Harle CA, Zhou X, Chen Y. Interactive systems for patient-centered care to enhance patient engagement. J Am Med Inform Assoc 2016; 23 (01) 2-4
• 21 Pal K, Eastwood SV, Michie S. , et al. Computer-based diabetes self-management interventions for adults with type 2 diabetes mellitus. Cochrane Database Syst Rev 2013; (03) CD008776
• 22 Marcano Belisario JS, Huckvale K, Greenfield G, Car J, Gunn LH. Smartphone and tablet self management apps for asthma. Cochrane Database Syst Rev 2013; (11) CD010013
• 23 Wieland LS, Falzon L, Sciamanna CN. , et al. Interactive computer-based interventions for weight loss or weight maintenance in overweight or obese people. Cochrane Database Syst Rev 2012; (08) CD007675
• 24 Rigotti NA, Clair C, Munafò MR, Stead LF. Interventions for smoking cessation in hospitalised patients. Cochrane Database Syst Rev 2012; (05) CD001837
• 25 Raniah Aldekhyyel, Michael Pitt, Caitlin Bakker, Genevieve Melton. The impact of interactive patient care tools used for the management of pain in an inpatient setting: a systematic review. PROSPERO 2018 CRD42018100105 Available at: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018100105 . Accessed July 24, 2019
• 26 Higgins JP, Lasserson T, Chandler J, Tovey D, Churchill R. . Methodological Expectations of Cochrane Intervention Reviews (MECIR) Standards for the Conduct and Reporting of New Cochrane Intervention Reviews, Reporting of Protocols and the Planning, Conduct and Reporting of Updates; 2016 . Available at: https://community.cochrane.org/sites/default/files/uploads/MECIR PRINTED BOOKLET FINAL v1.01.pdf . Accessed February 5, 2019
• 27 Clarivate Analytics, EndNote; 2016. Available at: https://clarivate.com/product-category/scientific-academic-research/ . Accessed July 21, 2019
• 28 Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic Reviews 2016; 5: 210 . Doi: 10.1186/s13643-016-0384-4
• 29 Moher D, Liberati A, Tetzlaff J, Altman DG. ; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6 (07) e1000097
• 30 Hoffman HG, Patterson DR, Carrougher GJ. Use of virtual reality for adjunctive treatment of adult burn pain during physical therapy: a controlled study. Clin J Pain 2000; 16 (03) 244-250
• 31 Sterne JA, Hernán MA, Reeves BC. , et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355 (October): i4919
• 32 Higgins JPT, Altman DG, Gøtzsche PC. , et al; Cochrane Bias Methods Group; Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011; 343 (02) d5928-d5928
• 33 Qualtrics. 2018. Avaliable at: https://www.qualtrics.com . Accessed July 21, 2019
• 34 Morris LD, Louw QA, Crous LC. Feasibility and potential effect of a low-cost virtual reality system on reducing pain and anxiety in adult burn injury patients during physiotherapy in a developing country. Burns 2010; 36 (05) 659-664
• 35 Tashjian VC, Mosadeghi S, Reid MW, Howard A, Lopez M, Spiegel B. Virtual reality reduces abdominal pain in hospitalized patients: results of a controlled trial. Gastroenterology 2017 Doi: 10.1016/S0016-5085(17)30727-8
• 36 Carrougher GJ, Hoffman HG, Nakamura D. , et al. The effect of virtual reality on pain and range of motion in adults with burn injuries. J Burn Care Res 2009; 30 (05) 785-791
• 37 Sharar SR, Carrougher GJ, Nakamura D, Hoffman HG, Blough DK, Patterson DR. Factors influencing the efficacy of virtual reality distraction analgesia during postburn physical therapy: preliminary results from 3 ongoing studies. Arch Phys Med Rehabil 2007; 88 (12) (Suppl. 02) S43-S49
• 38 Hoffman HG, Patterson DR, Carrougher GJ, Sharar SR. Effectiveness of virtual reality-based pain control with multiple treatments. Clin J Pain 2001; 17 (03) 229-235
• 39 Faber AW, Patterson DR, Bremer M. Repeated use of immersive virtual reality therapy to control pain during wound dressing changes in pediatric and adult burn patients. J Burn Care Res 2013; 34 (05) 563-568
• 40 Konstantatos AH, Angliss M, Costello V, Cleland H, Stafrace S. Predicting the effectiveness of virtual reality relaxation on pain and anxiety when added to PCA morphine in patients having burns dressings changes. Burns 2009; 35 (04) 491-499
• 41 Das DA, Grimmer KA, Sparnon AL, McRae SE, Thomas BH. The efficacy of playing a virtual reality game in modulating pain for children with acute burn injuries: a randomized controlled trial [ISRCTN87413556]. BMC Pediatr 2005; 5 (01) 1
• 42 Li XM, Yan H, Zhou KN, Dang SN, Wang DL, Zhang YP. Effects of music therapy on pain among female breast cancer patients after radical mastectomy: results from a randomized controlled trial. Breast Cancer Res Treat 2011; 128 (02) 411-419
• 43 Martorella G, Côté J, Racine M, Choinière M. Web-based nursing intervention for self-management of pain after cardiac surgery: pilot randomized controlled trial. J Med Internet Res 2012; 14 (06) e177
• 44 McDonall J, de Steiger R, Reynolds J. , et al. Patient activation intervention to facilitate participation in recovery after total knee replacement (MIME): a cluster randomised cross-over trial. BMJ Qual Saf 2019; pii:bmjqs-2018-008975
• 45 Wirz S, Conrad S, Shtrichman R, Schimo K, Hoffmann E. Clinical evaluation of a novel technology for oral patient-controlled analgesia, the PCoA® Acute device, for hospitalized patients with postoperative pain, in pilot feasibility study. Pain Res Manag 2017; 2017: 7962135
• 46 Interactive Patient Systems; 2017 Turning Interactivity into Outcomes - KLAS Report. Available at: https://klasresearch.com/report/interactive-patient-systems-2017/1234 . Accessed May 26, 2019
• 47 Aldekhyyel RN, Melton GB, Lindgren B, Wang Y, Pitt MB. Linking pediatrics patients and nurses with the pharmacy and electronic health record system through the inpatient television: a novel interactive pain-management tool. Hosp Pediatr 2018; 8 (09) 588-592
• 48 Aldekhyyel RN, Melton GB, Hultman G, Pitt MB. Using a bedside interactive technology to solicit and record pediatric pain reassessments: parent and nursing perspectives on a novel workflow. AMIA Jt Summits Transl Sci Proc 2018; 2017: 300-309
• 49 Rao-Gupta S, Kruger D, Leak LD, Tieman LA, Manworren RCB. Leveraging interactive patient care technology to improve pain management engagement. Pain Manag Nurs 2018; 19 (03) 212-221
• 50 Patmon FL, Gee PM, Rylee TL, Readdy NL. Using interactive patient engagement technology in clinical practice: a qualitative assessment of nurses' perceptions. J Med Internet Res 2016; 18 (11) e298
• 51 Martorella G, Côté J, Choinière M. SOULAGE-TAVIE: development and validation of a virtual nursing intervention to promote self-management of postoperative pain after cardiac surgery. Comput Inform Nurs 2013; 31 (04) 189-197
• 52 Westra BL, Johnson SG, Ali S. , et al. Validation and refinement of a pain information model from EHR flowsheet data. Appl Clin Inform 2018; 9 (01) 185-198
• 53 Gesner E, Collins SA, Rocha R. Pain documentation: validation of a reference model. Stud Health Technol Inform 2015; 216: 805-809
• 54 Westra BL, Christie B, Johnson SG. , et al. Modeling flowsheet data to support secondary use. Comput Inform Nurs 2017; 35 (09) 452-458
• 55 Genes N, Kim MS, Thum FL. , et al. Usability evaluation of a clinical decision support system for geriatric ED pain treatment. Appl Clin Inform 2016; 7 (01) 128-142
• 56 Couture B, Lilley E, Chang F. , et al. Applying user-centered design methods to the development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform 2018; 9 (02) 302-312
• 57 Roberts S, Chaboyer W, Gonzalez R, Marshall A. Using technology to engage hospitalised patients in their care: a realist review. BMC Health Serv Res 2017; 17 (01) 388
• 58 Careyva B, Shaak K, Mills G. , et al. Implementation of technology-based patient engagement strategies within practice-based research networks. J Am Board Fam Med 2016; 29 (05) 581-591
• 59 Carman KL, Dardess P, Maurer M. , et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood) 2013; 32 (02) 223-231
• 60 Sharma AE, Rivadeneira NA, Barr-Walker J, Stern RJ, Johnson AK, Sarkar U. Patient engagement in health care safety: an overview of mixed-quality evidence. Health Aff (Millwood) 2018; 37 (11) 1813-1820
• 61 Bates DW, Singh H. Two decades since to err is human: an assessment of progress and emerging priorities in patient safety. Health Aff (Millwood) 2018; 37 (11) 1736-1743
• 62 Schwappach DLB. Review: engaging patients as vigilant partners in safety: a systematic review. Med Care Res Rev 2010; 67 (02) 119-148