Keywords
user-centered design - patient safety - patient- and family-centered care - mHealth
Introduction
Efforts to make e-Health tools more usable for patients and care partners (i.e., family
and friends) have included a focus on the design of patient portals and mHealth applications.[1]
[2]
[3] mHealth “apps” initially gained traction as consumer products that were part of
the quantified self-movement and there is increasing recognition that mHealth apps
may aid in self-management in the ambulatory setting.[1] mHealth applications are increasingly being used in the hospital setting, yet there
is a need for more research around patient/care partner usability needs in the hospital
setting.[2] The Apple Research Kit is one example of a tool that is innovating ways in which
patient-facing research is conducted at major hospital institutions and offers useful
solutions that can be applied to many patient-facing apps (https://goo.gl/KYxVfG). Several academic medical centers have designed patient portals for the acute care
setting.[4]
The learning opportunities that e-Health tools for inpatient use open up for researchers
and health care systems are numerous and can help provide a new source of information
on patients and their care partners. Recent studies show that there is patient willingness
to participate in systems that allow them to communicate safety events.[5]
[6] Patients showed high levels of involvement in both participating in questionnaires
about safety and utilizing electronic tools to report safety concerns. However, both
studies showed that it is difficult to standardize data collection of patient-reported
safety concerns, and developed electronic tools are not always immediately generalizable
across a variable participant population. In developing e-Health tools, it is important
to remember that hospitalized patients have different characteristics than the general
population, and within a given population of hospitalized patients there can be wide
variation of socioeconomic status, baseline use of technology, and literacy and e-Health
literacy levels.[3] Care partners of hospitalized patients may experience emotional stress and increased
learning needs related to their loved one's hospitalization.[4]
[7]
[8]
[9] These emotional and situational factors add to the importance of conducting participatory
design when developing an mHealth app for hospitalized patients and their families.[10] e-Health literacy is related but different than general literacy levels, and furthermore
a patient's lowered health status when hospitalized could impact his or her ability
to interact with new technology.[11] In addition, appropriate use of consumer health terms are especially important for
any eHealth tools.[12]
[13] Participatory design with end-users may lead to better engagement, and more engaged
patients have been linked to better outcomes, shorter length of stay, and decreased
costs compared with less engaged patients.[2] Given these reasons, we identified the need to apply user-centered design methods
for the development of the patient- and care partner-facing user interface and functionality
of the MySafeCare Safety Reporting Application (MySafeCare [MSC] App).
We performed this study at a large academic medical center in the Northeast United
States. The MSC App is a Web-based and mobile-enabled safety reporting application
that facilitates real-time communication of concerns and worrisome events from hospitalized
patients and care partners to clinical staff. While there are several ways to capture
and address safety events identified by clinicians, there are minimal systemic efforts
and systems in place to learn more about safety issues from the patients' and care
partners' perspective.[5]
[6]
[14]
[15] Patient safety largely remains a clinician-centric topic despite evidence of a unique
patient perspective in detecting safety events.[14] A handful of systems exist, yet, overall, direct and real-time reporting from patients/care
partners in the hospital setting remains a critical gap in patient safety data sets
and was a motivation for the development and user-centered design of MSC.[5]
[6]
The MSC App has two components: a Patient and Carepartner Facing Component and a Clinical
Dashboard Component. The Patient and Family Facing Component is designed to run on
a mobile device, tablet, or laptop computer. It provides the user (patient, friend,
or family) with the option to identify themselves or remain anonymous, indicate their
type of concern through the 9 concern category icons, enter the level of severity
of the concern, and enter any free text to explain in their own words. See [Fig. 1] for screenshots of the final version of the Patient and Carepartner Facing Component.
The Patient and Carepartner Facing Component also includes a Compliments category
for compliments about staff or care received. The Clinical Dashboard displays the
submissions to appropriate clinical staff members (nurse and medical directors, patient
relations, and/or clinical staff). We have published prior work on the user-centered
design of the Clinical Dashboard.[16] This article describes the usability testing conducted with hospitalized patients
and care partners for the iterative design and refinement of the Patient and Carepartner
Facing Component of the MSC App. This study was approved by the hospital's Institutional
Review Board (IRB).
Fig. 1 Version 2 patient-facing app overall screenshots of three separate pages for easier
user navigation.
Methods
The user-centered design method allows for direct end-user engagement throughout the
design process, facilitating the development of a tool that can provide the most beneficial
outcome possible.[10] Part of this method includes conducting extensive usability testing on an initial
version to identify the enhancements needed around usability and functionality. An
initial MSC paper-based prototype was designed with a patient representative, patient
and family relations director, and clinical staff from our institution during a Hackathon
and refined into a wireframe prototype. This wireframe prototype was further refined
based on preliminary stakeholder interviews with Patient and Family Advisory Council
(PFAC) members (a hospital-sponsored council comprised of former patients and family
members aimed at improving patient experience) and clinical staff and used to develop
MSC Version 1. The following data sources informed the initial identification and
design of the 9 concern categories used in the App (e.g., medication, pain, infection
control, and medical device issues): (1) publications of patient-reported service
quality incidents,[17] (2) a hazard and near-miss reporting system,[18] (3) patient reports on service quality and safety, with a focus on waits and delays,
communication between staff and patients, and environmental issues and amenities,[19] and (4) preliminary stakeholder interviews with PFAC members and clinical staff.
All of this work produced Version 1 of the MSC App. The methods below describe the
user-centered design process performed on Version 1.
MySafeCare App Version 1 Usability Testing
We conducted individual scenario-based usability sessions, interactive group usability
sessions, and patient/care partner engagement rounds between February 2015 and September
2015 on Version 1 of the MSC App. The participants included hospitalized patients,
care partners, and members of PFAC. Targeted clinical units included the medical intensive
care unit (MICU), oncology unit, and intermediate vascular unit.
Individual Usability Sessions
Scenario-based usability sessions were conducted as 15- to 20-minute sessions in which
participants were asked to complete tasks in the MSC App for a set of predefined scenarios.
Patients who were alert and clinically stable (per their nurse) were asked to participate
and care partners, if present, were invited to participate as well. Given the level
of patient acuity in the MICU, only care partners completed the usability sessions
on the MICU. Scenarios were developed by research team members, including health systems
engineering and clinical experts, to reflect typical situations for patients while
allowing for systematic evaluation of specific tasks. Each scenario included two user
tasks. In total, 18 scenarios with 36 tasks were developed to enable sufficient testing
of each of the concern categories as well as other content within the application
(see [Table 1] for the main targeted functionality for testing and an example task). Each task
mapped to a specific feature that was being tested in the scenario. Each participant
was given one of the 18 scenarios. Two researchers were always present for the sessions:
one to conduct the session and the other to take detailed notes and record time of
completion for tasks.
Table 1
Types of features targeted for testing and example tasks
|
Main targeted features for testing
|
|
• Anonymous versus identified submission
|
|
• User type (patient, family member, or friend)
|
|
• Concern level
|
|
• Concern timeline
|
|
• Entering multiple concerns in one submission
|
|
Example tasks from usability testing
|
|
1. You want to remain anonymous
|
|
2. You are the patient
|
|
3. You are very concerned with your safety
|
|
4. Your safety concern happened more than 2 d ago
|
|
5. You have not shared your safety concern with the Care Team
|
|
6. You do not plan to share your concern with the Care Team
|
|
7. You are concerned that: “When people come into your room they always wear yellow gowns. Then someone comes
in without wearing a yellow gown”
|
|
8. If you choose to do so, please complete the background information about yourself
|
Participants were asked to “think-aloud” as they completed the scenarios on an iPad.
Using a predefined paper-based template to maintain consistency of the procedures
and types of data captured between participants, a researcher captured: participants
think-aloud comments, task completion successes and errors, instances when assistance
was required to complete the task, and amount of time required to complete the given
task.
Group Usability Sessions
Hour-long interactive group usability sessions were also conducted with the hospital's
PFAC. PFAC participants were shown the MSC App and were given two scenarios to walk
through to simulate submitting a concern. All participants were given the same scenarios
to facilitate group discussion after completing the submission. Detailed notes of
the dynamic interactive group discussions were recorded by the research team.
Engagement Rounds
Finally, we conducted biweekly engagement rounds to patient rooms and family waiting
room areas from April 2015 to October 2015. MSC App Version 1 was available for use
on clinical units during this period, while iterative design and development continued
to inform refinements for Version 2. Engagement rounds were conducted to inform patients
and care partners of the availability of the MSc App, how to access it, and answer
any additional questions. Patients and care partners that were interested and engaged
would often ask to use the app right then. Any questions, concerns, issues, or difficulties
that came up during these engagement rounds were noted by the team members and triangulated
with data from our individual and group usability testing to further inform iterative
development.
Analysis of Usability Work and Theme Development
Three steps, each with several substeps, were used to identify specific requirements
for application revisions: (1) Theme Development: (a) compilation of all notes from usability sessions and engagement rounds, (b)
coding of discrete issues, (c) grouping of discrete issues into categories, (d) identification
of theme for each category; (2) Association to Nielson Heuristics: (a) user-driven requirements and identified issues, categories, and themes were
discussed for group consensus and mapped to Nielson Heuristics during team meetings;
(3) Versioning of the MSC App: (a) translation of MSC App issues for each theme into specific requirements to inform
the development of Version 2 of the MSC App, (b) implementation of specific requirements
for Version 2, and (c) continuation of the iterative user-centered design process
for new version of the MSC App.
Detailed documentation from the usability testing sessions and engagement rounds—particularly
difficulties or issues users experienced when completing the scenarios and “think-aloud”
comments—were analyzed in depth. Two research team members performed the thematic
qualitative coding of the data with one team member as the primary coder performing
the initial coding and the second team member refining and validating the coding.
The coded themes were discussed and confirmed with the larger research team for consensus.
QSR International's NVivo 10 software for qualitative data analysis was used to capture
and organize the results. Data collection continued until the iterative analysis of
issues indicated data saturation had been reached. Nielsen's 10 heuristics identify
general guidelines for developing a user-friendly tool; the themes identified in our
usability analysis of the MSC App were evaluated in the context of how they related
to Nielsen's 10 Usability Heuristics.[20]
Results
MySafeCare App Version 1 Usability Testing
A total of 11 individual usability sessions, 3 small group usability sessions with
25 members of PFAC, and over 250 interactions with patients and families during engagement
rounds were conducted to inform usability of the Patient and Carepartner Facing Component
of the MSC App. Each PFAC session had on average 6 to 10 participants, and consisted
of a combination of former patients and care partners. Over the course of 7 months,
up to 286 participants interacted with the MSC App.
Analysis of Usability Work and Theme Development
Twelve discrete issues were categorized during analysis of the user-centered design
sessions on Version 1 as necessary revisions for Version 2 of the MSC App. These were
further categorized into three distinct themes which fell under the headings of terminology,
workflow functionality, and user interface design ([Table 2]).
Table 2
Identified Version 1 application issues, associated themes, and example revisions
for Version 2
|
Themes
|
Issues
|
Example revisions
|
|
Workflow functionality
|
1. Informed consent text in introduction is too long to read/understand
2. Need to capture if user “Accepts” or “Does not accept” the informed consent information
3. When entering more than one concern in a single submission it is not intuitive
to click the “plus” button. Additional functionality is needed to associate each concern
with a severity level
4. A category to submit a compliment should be added
|
For issue #3:
Logic revised so that user is prompted to choose either a Concern or Compliment which
automatically navigates to appropriate submission page Submission page includes buttons
labeled “Add Another” and “Remove” to allow for additional submissions or modifications
by the user
|
|
Terminology
|
The term “safety concern” is confusing. A phrase that is better understood by users
is “worrisome or concerning event”
Concern category labels are confusing. The addition of “My” is clearer to users that
the categories relate to their care and experience
The use of the following terms in the app generated confusion: “caregiver,” “medical
device,” “infection” category, “self.” More clear and directed terminology includes
“family, “hygiene,” and “I am the patient”
4. Specification of how to answer the optional background questions is needed based
on user type (patient, family, friend)
|
For issue #1:
All references to “safety concern” were revised to “worrisome or concerning event,”
including consent forms, posted flyers, and engagement rounds discussion. Additionally,
the question “Would you say your stay has gone perfectly? If not, we would like to
know” was added to the first page to convey that we are seeking a broad spectrum of
safety and quality issues
|
|
User interface design
|
1. “Minus” and “Plus” buttons for adding/deleting a concern are not labeled and therefore
hard to find and understand their function
2. The icons and labels for the concern categories are too small and hard to read/see,
especially on smaller devices
3. The hospital logo is not in an obvious position (off to the side), and there is
too much extra white space on the introduction page
4. All contents of the application are on one long page, which requires cumbersome
scrolling and often results in users missing sections of the application
|
For issue #4:
The application was modified to stratify content across 7 short pages based on the
natural grouping of questions to limit the amount of scrolling required of users
|
After review of data and discussions to achieve group consensus with research team
members, the final high-level themes identified as major design implications when
creating a mobile application for patient/family use in an acute care hospital setting
were:
-
Terminology to match hospitalized patient and family health literacy and shared understanding.
-
Simplified intuitive workflow functionality for patients and families who may experience barriers to navigating apps while in
the hospital setting.
-
User interface design for mobile devices.
[Fig. 2] shows the four heuristics that we found were specifically identified by hospitalized
patients and care partners as needing refinement during our in-depth participatory
design: match between system and the real world, consistency and standards, flexibility
and efficiency of use, and aesthetic and minimalist design.
Fig. 2 MySafeCare as an mHealth tool compared with Nielsen's Usability Heuristics.
Workflow Functionality
Three issues associated with the workflow of the MSC App were highlighted during both
the individual and group usability testing sessions. The scenario walk-through was
challenging for users of Version 1 when trying to enter more than one concern at a
time. Specific issues included that end-users had difficulty in navigating to the
“plus” and “minus” buttons that denote adding or removing a concern, and end-users
were confused how to choose a severity level when more than one concern was involved.
These issues required revision to the sequence of items presented to the end-user
and branching logic for a simpler and more intuitive workflow. [Fig. 3] illustrates these issues on Version 1 and the updated Version 2.
Fig. 3 MySafeCare Patient and Family Facing Component–Version 1 with annotations for refinement
based on user feedback, and Version 2.
Terminology
During engagement rounds, our research team members used the term “safety concern”
to describe to the purpose of the MSC App to patients and care partners. However,
this was consistently met with confusion; “safety concern” was interpreted as an event
in which actual harm had occurred, while the intent of the MSC App is to capture experiences
in which the patient or family felt threatened and could potentially lead to a harmful
event. The phrase “worrisome or concerning event or situation” was identified as the
best phrase to reduce this semantic mismatch. Through usability testing, we found
that the content of Version 1 of the application did not sufficiently account for
health literacy levels of the patient and family population. One concern, category
labeled Medical Device, raised confusion since not all patients or care partners are
aware of what is considered a medical device. This was reconfigured into the “My Room”
category to more accurately capture safety issues relating to a patient's immediate
surroundings.
[Table 3] depicts the initial Version 1 concern categories and the final Version 2 categories
and subcategories. Notably, “Medical Device” was changed to “My Room,” and “Infection”
was changed to “My Hygiene.” The use of “My” in each concern category is one example
of a finding and design change from a PFAC group session aimed at personalizing the
application for users so that they understand this is asking about their experiences.
Both PFAC and patient/care partner participants also confirmed that a free text box
should be available for users to explain the concern in their own words.
Table 3
Final categories, subcategories, and icons for the MySafeCare App Version 2
|
Version 1
|
Version 2 (Final)
|
|
Concern categories
|
Concern categories
|
Icon images
|
Associated subcategories
|
|
|
|
I/my family caregivers don't know my plan of care
|
|
|
I/my family caregivers feel like my care team isn't following my plan of care
|
|
Plan of Care
|
My Plan
|
I/my family caregivers feel like there is a problem with my plan of care
|
|
|
I/my family caregivers have other treatment concerns (please explain in box below)
|
|
|
|
I was given the wrong medication or dose
|
|
|
I was almost given the wrong medication or dose
|
|
Medication
|
My Medication
|
I was not given my medication on time
|
|
|
I missed a medication
|
|
|
I/my family/caregivers have other medication concerns (please explain in box below)
|
|
|
|
My medical device is not working
|
|
|
My medical device will not stop beeping
|
|
Medical Device
|
My Room
|
My medical device seems excessive
My room is not clean
|
|
|
I/my family caregivers have other medical device concerns (please explain in box below)
|
|
|
|
I/my family caregivers don't feel respected
|
|
|
My and/or my family caregivers' needs are being ignored
|
|
Communication
|
My Communication
|
I/my family caregivers am/are concerned about the communication between my care team
about my plan of care
|
|
|
I/my family caregivers am/are concerned about how my care team communicates with me
about my plan of care
|
|
|
I/my family caregivers have other communication concerns (please explain in box below)
|
|
|
|
A member of my care team did not wash his/her hands
|
|
|
A member of my care team did not wear gloves
|
|
Infection
|
My Hygiene
|
A member of my care team did not follow the precautions on the door
|
|
|
|
|
|
I/my family caregivers have other infection concerns (please explain in box below
|
|
|
|
My and/or my family caregivers' privacy is/are being ignored
|
|
Privacy
|
My Privacy
|
I/my family caregivers have other privacy concerns (please explain in box below)
|
|
|
|
I/my family caregivers feel that my care team is not managing my pain to my expectations
|
|
Pain
|
My Pain
|
My pain is well controlled but I/my family caregivers am/are concerned about the medication
|
|
|
Nobody has asked me or my family caregivers if I am in pain
|
|
|
I/my family caregivers have other pain management concerns (please explain in box
below)
|
|
|
|
I am waiting too long for help going to the bathroom
|
|
|
I am waiting too long for help turning and moving in bed
|
|
|
I am waiting too long for my procedure
|
|
Waiting Time
|
My Waiting Time
|
I am waiting too long to be transferred
|
|
|
I am waiting too long to be discharged
|
|
|
I/my family caregivers have other waiting time concerns (please explain in box below)
|
|
Other
|
Other
|
|
|
We also observed variation in how patients and care partners categorized safety concerns
from the provided scenarios. Specifically, 6 out of the 11 participants that took
part in the individual usability testing sessions selected a different concern category
for a given scenario than what was intended by the research team when drafting the
scenario. For example, a given scenario was: “Your mother who visits frequently doesn't understand your care plan.” This was intended to fall under the category “Plan of Care (My Plan in Version
2)”; however, the patient interpreted it as a Communication issue. Another scenario
was: “When people come into your room they always wear yellow gowns. Someone came in today
without wearing a yellow gown.” This was intended to be an Infection concern (My Hygiene in Version 2), but was
entered under the Plan of Care (My Plan in Version 2) concern category. This mismatch
in interpretations highlights the need to learn more about patient and care partner
perceptions on safety and what they deem threatening. It also highlights that a given
safety concern can be categorized into more than one categories, and these categorizations
may be perceived differently by different individuals. This finding emphasized the
need for the free text box to allow the user to express the concern in his or her
own words to ensure better communication of the issue at hand.
User Interface Design
Some of the user interface problems that were identified included the visibility of
concern category icons, appropriate labeling of navigation features, and the amount
of scrolling required to complete the submission. Version 1 was formatted to have
all content on one page, which included lengthy informed consent information, questions
to identify the user, concern submission content, and optional background questions.
While this approach was intended to decrease the need for users to navigate multiple
pages, the usability testing feedback identified that it was cumbersome. This intensive
scrolling was an issue for two reasons: (1) some users were unaware they had to scroll
for more content and may have stopped without knowing how to proceed if a research
team member was not present to point it out, and (2) patients that reported they were
particularly ill, tired, or had just undergone testing or treatment of some sort often
stated that they did not have the energy for the level of reading and scrolling that
was required to get through the MSC App. Version 2 was redesigned with page breaks
based on user feedback, implemented at natural partitions in application content so
that each page required minimum or no scrolling with the “Next” button clearly visible.
Overall screenshots of versions 1 and 2 of the MSC App are shown in [Figs. 1] and [4], respectively. Version 1 is all one long page, whereas in Version 2 each screenshot
represents a different page.
Fig. 4 Version 1 patient-facing app overall screenshots of single page requiring scrolling.
Discussion
The hospital can be a very stressful environment for both patients and their family.
Introducing new elements like the MSC App requires analysis of, and integration with,
the users and their specific needs and restrictions. The three major high-level themes
identified through usability testing, restated below, reflect on the importance of
end-user engagement and may be applied to other types of mHealth applications for
hospitalized patients:
-
Simplified intuitive workflow functionality for patients and families who may experience
barriers to navigating apps while in the hospital setting.
-
Terminology to match hospitalized patient and family health literacy and shared understanding.
-
User interface design for mobile devices.
In our usability testing, we identified key issues and achieved data saturation quickly
in identifying the types of challenges and struggles users experienced with Version
1. Patients and care partners identified issues and suggested better approaches to
improve the design, confirming the value and need for participatory design.
The workflow issues we identified aligned with Nielsen's flexibility and efficiency
of use heuristic. For example, users identified challenges to entering more than one
Compliment or Concern and general navigation of the application. In response, we revised
the logic configured within the app to provide more intuitive and efficient navigation
consistent with end-user preferences, while still allowing the user the flexibility
to enter multiple concerns or compliments and revise prior data entries.
Our user interface design changes aimed to enhance the aesthetic and minimalist design
of the app to decrease user burden for patients and care partners with varied levels
of comfort with technology and levels of illness. The user interface issues we identified
likely apply to mobile app development overall, as confirmed by design guidance published
by Apple Research Kit.[21] We identified major usability issues in Version 1 due to excessive scrolling required
of the user to navigate the application. One major barrier to use was the length of
the informed consent text required by the IRB to inform users that MSC is a research
study and provide guidance to the user in case of a serious safety concern or incident.
Participants noted that patients who were sicker would not have the energy to read
the lengthy text. In response, we collaborated with the IRB to revise the text and
provided a link to a video we created communicating the informed consent information.
We also mimicked the style of common mobile apps through pagination of different sections[21] based on the types of questions, resulting in a few questions per page to minimize
scrolling, and applied branching logic to the questions to decrease user burden.
Notable, there are a few differences between the MSC Safety Reporting Tool and other
research apps targeted at patient users that we are aware of, specifically relating
to the informed consent text and account creation upon initial use of the app. Account
creation is typical on many other research apps (e.g., Apple Research Kit), and users
of Apple Research Kit apps can bypass the informed consent information after the initial
login. With MSC, patients and care partners may submit anonymously, therefore we do
not capture user signatures or create accounts (in the app or for IRB consent) to
preserve anonymity of participants. The requirement that patients/families trusted
that their anonymity was preserved informed our approach to use a Web-based platform
instead of requiring downloading to a device, as is typical of apps. We found that
provision of all required informed consent information while designing anonymous reporting
applications that are intuitive, easy to navigate, and quick is challenging, but possible.
We expect that other patient-facing apps may have similar requirements and recommend
future work to identify best practices for mHealth apps that garner patient trust
and are enabled to preserve anonymity while meeting other research or operational
needs.
Making sure that information and terminology is consistent and understandable across
varying users is essential to the success of an application and our findings reflect
Nielsen's Heuristics of “match between system and the real world” and “consistency
and standards.” MSC targets “near-miss” and “unsafe condition” cases as defined by
the Agency for Healthcare Research and Quality (AHRQ) Common Format[21]; in other words, situations/events that are worrying and concerning to patients
and care partners have the potential to turn into actual safety events, and could
be mitigated in real-time and used to improve our safety systems overall.[14] Tailoring the phrasing and word choice of the application content to effectively
describe the intent of the application to the end-user is a major priority. The semantic
mismatch we observed in patient/care partner understanding of the phrase “safety concern”
is a great example of how there can be impactful differences in interpretation between
clinical/research staff and patients and care partner, in this case around patient
safety concepts. We learned that when a patient hears “safety” they think “physical
harm” that has already occurred and tended to focus primarily on medication-related
events and not consider other types of near misses, such as communication concerns.
We found our participatory design approach, particularly our patient and care partner
engagement rounds, essential to understand how best to convey that MSC intends to
capture incidents not only after they occur, but at any point in time in which an
experience as a patient does not seem appropriate. These engagement rounds complemented
the scenario-based usability sessions to identify themes from a large sample of hospitalized
patients, and allowed for observation of how well the application was being used—and
understood—“in the wild.” We note that struggles observed during usability sessions
could be magnified for patients that are more acutely ill or that have just undergone
a treatment, thus highlighting the importance of this method for designing an electronic
tool for use in the hospital setting.
Based on our findings, we recommend that participatory user-centered design of mHealth
apps for patients and care partners in the hospital setting particularly emphasize
four heuristics: match between system and the real world, consistency and standards,
flexibility and efficiency of use, and aesthetic and minimalist design. We found the
process of addressing these four heuristics with end-users improved design aspects
aligned with all of Nielsen's heuristics, and this should be confirmed with future
work. For example, simplifying app design workflow to provide flexibility and efficiency
of use, such as including easier navigation between pages, may also enable users to
recover if they become lost or enter data in error and may increase visibility of
system status. A key principal in our design was to make MSc efficient and flexible
to balance the collection of reliable, structured, coded data while not preventing
partial data collection that included enough information for a real-time response
or continuous learning. We emphasized efficiency by utilizing short, structured questions
with predefined coded values, but also made these questions optional and allowed for
unstructured, flexible data entry in free text form, and in doing so provided control
and freedom to users. Finally, Nielsen's “recognition rather than recall” heuristic
is interesting to note from our study's perspective because while it is important
in all systems, it could be considered a defining design characteristic and assumed
feature of an app targeted at infrequent or one-time end-users with no expectation
of training or baseline knowledge, such as hospitalized patients.
A final noteworthy challenge is how to make hospitalized patients aware of new initiatives
related to patient engagement. Based on this study and prior studies, our team has
learned that providing viewable information in the patient room to notify patients/families
is useful when combined with the conduct of daily engagement rounds.[22]
[23] Patient engagement in the hospital setting represents a challenging and evolving
area within implementation science.
Limitations
A limitation of this study was that usability sessions were not audio or video recorded,
which may have allowed for deeper analysis of think-aloud comments. Video recording
was out of the scope of this study (our devices were Apple iPads which do not allow
for screen recording), though may have added to our data collection on errors and
time spent navigating the application. However, all sessions were conducted with two
researchers, with one exclusively responsible for detailed note taking and time keeping.
Additionally, systems for patient-reported safety concerns will have an inherent limitation
in that interpretation of safety issues is complex. Safety issues are very often multifactorial,
so the design of a system to capture concerns is a learning process. Finally, while
usability testing did not test users' ability to access the app on their own device,
we did observe patients/care partners accessing the app on their own during engagement
rounds.
Conclusion
Three major themes resulted from in-depth participatory design work regarding major
areas to address when developing a mobile patient safety app for use in a hospital
setting: workflow functionality, terminology, and user interface design. Workflow
issues highlighted functionality in the app that were not intuitive or easy to navigate
for hospitalized patients. We identified terminology issues that are likely specific
to the domain of patient safety from the patient and care partners' perspective. The
user interface design issues identified were consistent with existing guidelines for
mobile applications, confirming the dependability of our findings.[23] Future work should investigate if similar themes surface during user-centered design
of other patient-facing applications.
We illustrated that the issues identified for these themes specifically aligned with
four of Nielsen's Usability Heuristics: match between system and the real world, consistency
and standards, flexibility and efficiency of use, and aesthetic and minimalist design.
While several user issues identified may not be limited to the inpatient population,
the experience of being acutely ill in the hospital may enhance barriers to use. Our
findings support “in-the-wild” testing in designing an mHealth app intended for use
by patient and care partners in the hospital setting. We recommend more research to
determine best practices and guidelines for apps targeting these populations. Our
future work includes further optimization of MSC through implementation and evaluation
across different clinical units to collect usability feedback from varying patient
populations.
Multiple Choice Questions
Multiple Choice Questions
-
Which of the following related to the patient and care partner perspective of care
concerns and safety reporting is true:
-
A patient or family perspective of safety threats may differ from a clinician's perspective
of safety threats
-
Fear of retaliation never occurs for patients or families that have safety incidents
or threats they want to report
-
Only patients and families with low health literacy need to be educated about how
to report safety threats
-
Care concerns from the patient and family perspective are narrow in scope and are
not safety threats
Correct Answer: The correct answer is option a. Patients and families have a unique and valuable
perspective of care concerns and safety threats and these perspectives may differ
from clinicians' traditional perspectives of patient safety.
-
Which of the following are key criteria for design of mHealth applications used by
patients and care partners in the hospital setting?
-
Simple, intuitive design and language that fits literacy level requirements
-
Requires user training conducted by a full-time devoted IT resource
-
Includes medical abbreviations and terminology used by clinicians
-
Identifies patients and care partners through a unique identifier
Correct Answer: The correct answer is option a. Hospitalized patients and family members are under
emotional stress, which can lead to increased learning needs, therefore necessitating
simple mHealth tools with low learning curves and simple, direct language. Additionally,
patients and family members may have a wide range of literacy and health literacy
levels.
