Keywords
electronic patient portal - health information management - chronic disease - workload
- usability - task-technology fit
Background and Significance
Background and Significance
As a result of the HITECH Act of 2009, which incentivized eligible hospitals and providers
to make electronic data directly available to patients,[1] electronic patient portals are now offered by most health care organizations and
are becoming more popular.[2]
[3] Portals can provide individualized information to patients on their medical conditions
and medications, laboratory results, their care teams, their hospital stays, and their
expected care plans. These digital communication solutions usually also allow for
secure, asynchronous patient–provider communication, and other functions such as refilling
medications, scheduling appointments, or paying bills.[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18] People who use e-health resources feel better prepared for clinical encounters,
ask more relevant questions, know more about their health care, and are more likely
to take steps to improve their health.[19] Limited evidence has also linked the use of portal features to improved outcomes
for chronic conditions such as diabetes, depression, and hypertension.[20]
[21]
[22]
[23] Over the long term, it is also hoped that portals might engage patients and improve
patient activation, which is associated with adoption of healthy behaviors, better
disease self-management, increased health information seeking,[24] and better health outcomes.[25]
However, despite these potential positive outcomes, it is well established that racial/ethnic
minorities and patients with limited health literacy, income, and education are significantly
less likely to use portals.[26]
[27]
[28]
[29]
[30]
[31] Although the overall use of digital health information management tools has increased,
a segment remains disengaged.[32] As described in the DeLone and McLean model, an information system cannot be expected
to produce its desired outcomes unless it is perceived as usable and contributes to
user satisfaction.[33] Usability (the capacity of the system to allow users to carry out their tasks safely,
effectively, efficiently, and enjoyably[34]) has been reported as a barrier to use of patient portals.[35]
[36]
[37]
[38]
[39]
[40]
[41] Usability is important to users with high levels of education and routine Internet
use, who want applications that meet usability standards similar to those in retail
industries.[32] It may be even more important to users with less education and computer expertise,
who have less awareness of technology and limited technology skills.[32]
[42]
[43]
[44] Design features that improve usability and perceived organization and clarity influence
patient portal adoption.[45]
[46] Specific usability barriers in portals include difficulty navigating through functions,
confusing terms, poor display of information, confusing functionality, and long times
to perform a task.[41]
[47]
[48]
Usability is often studied as a function of an individual user interacting with the
technology interface. However, as multiple researchers have previously pointed out,
patient portals and other information technologies are used by patients in the context
of the work that their medical conditions impose.[49]
[50]
[51] Usability can be improved only with an understanding of the tasks that patients
need to perform in the course of their “illness work.”[52] We therefore approached usability for patients through the lens of task-technology
fit models,[53] specifically Ammenwerth et al's “fit between individuals, task, and technology”
(FITT) framework[54] ([Fig. 1]). According to FITT, technology adoption depends on the fit between the attributes
of the users (e.g., skills, motivation, and knowledge), the attributes of the technology
(e.g., usability, functionality, performance), and the attributes of the tasks (e.g.,
their organization and complexity).
Fig. 1 Adapted from Ammenwerth's fit between individuals, task, and technology (FITT) model,
our conceptual model links electronic patient portal adoption with fit among personal
health information management (PHIM) tasks, patients, and the portal. In this study,
we assessed five PHIM tasks and collected information about patients, the portal,
patient-task fit, patient-portal fit, and task-portal fit.
In previous work, we had identified types and characteristics of personal health information
management (PHIM) tasks that patients with chronic illnesses do as part of their illness
work.[49] Some of the tasks described most frequently included checking/tracking personal
medical indicators such as laboratory results, sharing medical record information
from one doctor to other health care providers, and searching for background medical/health
information.[49] In the current project, we evaluated an electronic patient portal to determine how
easily it could be used by diverse patient populations to support these sorts of PHIM
tasks. We collaborated with a large academic hospital in New York City, New York-Presbyterian,
which had recently relaunched an electronic patient portal and was interested in improving
its usability. The portal, www.myNYP.org, provides patients with inpatient data such as laboratory results, procedures, and
care instructions after their hospital discharge, and is currently available in English,
Spanish, and Mandarin versions. (An additional feature that we did not evaluate was
an inpatient segment that delivers up-to-date information to patients during their
hospital stay.[55])
Objectives
Our objectives were to identify task-technology fit problems and usability challenges
in the novel portal, recommend solutions, and to evaluate whether the recommended
design changes improved usability. We conducted this study in three stages. First,
we applied heuristic usability testing to explore the technology and identify and
remediate obvious usability problems. Second, we conducted user testing to assess
fit between the PHIM tasks and the electronic patient portal (task-technology fit)
as well as fit between patients and PHIM tasks (patient-task fit) and between patients
and the technology (patient-technology fit, specifically perceived usability); specific
barriers to the PHIM tasks were presented to the development team for remediation.
Third, we conducted a final round of user testing to determine how well the changes
improved task-technology fit and patient-technology fit (perceived usability).
Methods
Phase 1
We launched this project with a heuristic usability evaluation, because this technique
is an efficient way of identifying characteristics of the technology and remediating
obvious usability barriers before time-intensive user testing is started.[56] Before the portal went live, three evaluators applied a heuristic usability checklist
developed by Zhang et al (built upon work by Nielsen and Schneiderman).[56] Each evaluator individually analyzed the portal, identified heuristics that were
violated, and assigned a severity score to each heuristic that was violated. After
several rounds of discussion, a consensus usability score was reached. Based on the
observations and consensus scores, a report of recommendations on improvements was
presented to the portal development team. After the development team selected recommendations
to implement on the basis of feasibility and institutional priorities, heuristic evaluation
was conducted a second time using the same procedures.
Phase 2
Phase 2 was user testing in which users were invited to perform typical health information
management tasks[49] and to determine how well they could use the portal to perform these tasks.
For user testing, inclusion criteria were: 18 to 95 years of age; ability to speak
in English; ability to use a computer and navigate through Web sites; and either having
a chronic medical condition or being a caregiver for a person with a chronic medical
condition. Patients were recruited with multiple methods, including flyers in hospital
clinics, an online sign-up form on the existing patient portal, and direct approach
in waiting areas. Patients who met the inclusion criteria and were interested in participating
were introduced to the study using a standardized script. Informed consent was obtained.
At the end of the session, the patient was given a gift card worth US $10. The project
was approved by the Weill Cornell Institutional Review Board.
Participants were invited to log in to the portal using a fictitious account that
was prepopulated with patient data. The participants were first invited to explore
the portal for its various functionalities and affordances, then invited to complete
five tasks chosen to be representative health information management tasks[49]:
-
Reviewing personal information from a previous hospitalization.
-
Creating a report to be given to a physician at another organization.
-
Reading up about a specific medical condition.
-
Changing the account password.
-
Enabling a family member's access to the account (proxy access).
Screen activity was recorded through an audio recording and screen-capture software
(Morae, TechSmith, Inc., Okemos, Michigan, United States). An additional layer of
rich qualitative data was collected by inviting the participants to provide a continuous
verbal accounting of their thoughts as they completed the tasks, a procedure known
as the concurrent “verbal protocol” or “think-aloud protocol” procedure. These protocols
have been demonstrated to capture cognitive processes during problem solving.[13] In addition to performing each task, participants were invited to comment on how
well the technology allowed them to perform the task. Audio recordings were transcribed
and analyzed.
Sampling was wrapped up after 12 participants, when data analysis (described below)
suggested we had achieved saturation (new interviews were not producing substantively
new themes).[57]
[58] However, after we concluded sampling, we excluded one interview because the participant's
demographic questionnaire showed that the participant did not meet all the inclusion
criteria. Our final totals throughout the rest of the paper therefore refer to 11
participants in this phase.
At the end of the user testing session, participants were invited to complete a demographic
questionnaire, a standard usability questionnaire,[15] the Single Item Literacy Screener (SILS),[59] and the Subjective Numeracy Scale (SNS), a self-report measure of quantitative ability
and preferences.[60]
Phase 3
Phase 2 recommendations were presented to the portal development team, and after selective
implementation, an additional group of users was recruited to conduct user testing,
using the same procedures and interview guide. We targeted 12 users for this phase
to match the 12 that had been interviewed in the earlier phase.
Data Analysis
Data from user testing underwent two types of analysis. First, Morae videos were reviewed
to assess whether each task was completed by the participant (task-technology fit;
[Fig. 1]). Tasks were categorized as completed easily; completed with assistance; not completed;
or replaced with different task to accomplish the goal. An example of this substitute
task was taking a screenshot of laboratory values to share with a doctor rather than
creating a continuity of care document (CCD) report.
In a clinical setting, such substitute tasks are generally classified as “workarounds,”
nonstandard procedures used by professionals to accomplish a task in situations when
deficiencies in system design create barriers to using the standard procedure.[61] However, in our observations, it was clear that in many cases the patients did not
understand the health information task in the first place, and specifically did not
know the standard/recommended procedure for accomplishing the task. Given these ambiguities,
we opted to call them “substitute tasks” instead of workarounds.
In the second part of the data analysis, audio recordings of the concurrent verbal
protocols were transcribed and coded qualitatively to assess task-technology fit,
user-technology fit, and user-task fit ([Fig. 1]). The first two transcripts were reviewed by three coinvestigators (J.S.A., S.A.,
B.H.) to develop the initial codebook. After that, each transcript was coded independently
by at least two and in some cases three of the researchers, who met to establish consensus
on each. As the Ammenwerth et al FITT framework[54] focuses on the relationship between the technology, the task, and the user, we developed
the initial codebook to focus primarily on mismatches between the user and the technology and the user and the task ([Fig. 1]). A third category, mismatches between the technology and the task, was also used to identify occasional Web site bugs, which are not presented here
but were presented to the portal development team for remediation. Additional codes,
such as the emotional valence expressed by the participants, were developed through
inductive thematic analysis.[62]
System usability scores, which did not satisfy the Shapiro–Wilk test for normality,
were compared with the Wilcoxon rank-sum test, using R version 3.4.1.
Results
Heuristic Evaluation
In the pre-go-live heuristic evaluation ([Fig. 2]), the most severe usability barrier identified was Failure to use users' language, with a severity score of 4 out of possible 5. An example was that instead of inviting
patients to create a report of their medical information, the Web site invited patients
to click a button labeled “CCD export.” The most severe potential error was the availability
of an option for patients to accidentally delete the entire medical record within
the portal.
Fig. 2 Heuristic evaluation, performed by the researchers, showed gains in some domains
as well as losses in others. Losses were largely attributable to the introduction
of new functionality and reorganization during the development phase.
The portal development team implemented several recommendations that were feasible,
could be accommodated with developer workload, and met institutional priorities. However,
simultaneously new functionality was implemented, and functions were reorganized.
For example, a bill-paying function was updated and replaced. As a result, even after
revisions, the second heuristic evaluation found that Failure to use users' language remained the most severe problem with a score of 4. After the first evaluation, the
most severe errors were corrected and additional explanatory material was added, creating
improvements in Error prevention and Help and documentation. Conversely, the additional content meant that three domains scored worse the second
time around. (Additional improvements and updates were made after the second heuristic
evaluation, as part of the ongoing development cycle.)
User Testing
In the user testing phases, the 23 participants represented a range of ages, races
and ethnicities, and insurance categories ([Table 1]). Participants had chronic conditions including type 1 diabetes, type 2 diabetes,
and cancer, or were caring for family members with conditions such as ulcerative colitis
and thalassemia. Only two reported needing assistance with medical documents (it is
likely that patients with lower levels of literacy were screened out because of inability
to use computers). Self-reported numeracy ranged from 17 to 48 on the scale of 8 (low
numeracy) to 48 (high numeracy); 38% reported a score of 30 or less.
Table 1
Participant demographics
|
Group 1
|
Group 2
|
|
N
|
%
|
N
|
%
|
|
Number of females
|
10
|
83
|
10
|
83
|
|
Age category
|
|
|
|
|
|
18–40 y
|
7
|
58
|
6
|
50
|
|
41–64 y
|
4
|
33
|
6
|
50
|
|
65+ y
|
1
|
8
|
0
|
0
|
|
Education level
|
|
|
|
|
|
High school grad or GED
|
3
|
25
|
3
|
25
|
|
Some college
|
2
|
17
|
3
|
25
|
|
College degree
|
4
|
33
|
4
|
33
|
|
Master's degree or higher
|
3
|
25
|
2
|
17
|
|
Race/ethnicity
|
|
|
|
|
|
White
|
9
|
75
|
3
|
25
|
|
Hispanic/Latino
|
0
|
0
|
5
|
42
|
|
Asian
|
2
|
17
|
1
|
8
|
|
African-American
|
0
|
0
|
1
|
8
|
|
Other
|
1
|
8
|
0
|
0
|
|
No response
|
0
|
0
|
4
|
33
|
|
Insurance coverage
|
|
|
|
|
|
Private insurance
|
5
|
42
|
4
|
33
|
|
Medicare
|
2
|
17
|
1
|
8
|
|
Medicaid
|
3
|
25
|
2
|
17
|
|
Other
|
2
|
17
|
0
|
0
|
|
Marital status
|
|
|
|
|
|
Married
|
6
|
50
|
6
|
50
|
|
Not married
|
6
|
50
|
1
|
8
|
|
No response
|
0
|
0
|
5
|
42
|
|
Single Item Literacy Scale
|
|
|
|
|
|
Any literacy problem
|
1
|
8
|
1
|
8
|
|
No literacy problem
|
6
|
50
|
8
|
67
|
|
No response
|
5
|
42
|
3
|
25
|
|
Subjective Numeracy Scale
|
|
|
|
|
|
Below 30 (lower numeracy)
|
4
|
33
|
8
|
67
|
|
30+ (higher numeracy)
|
5
|
42
|
7
|
58
|
Abbreviation: GED, General Educational Development.
Note: Numbers do not sum to 100% because of rounding.
When invited to complete the five tasks, changing the account password was the only
task that was completed easily by almost all of the participants ([Fig. 3]). This was probably because for account settings, standards were familiar from Web
commerce sites.
Fig. 3 Patients' ability to complete several personal health information management (PHIM)
tasks improved after portal redesign.
User performance on most other tasks improved noticeably in phase 3, after design
changes suggested in phase 2. At this stage, most participants were able to identify
and review data from previous hospitalizations (task 1), and create a report for an
external physician (task 2). In phase 2, many patients had difficulty finding the
medical encyclopedia resource because it was labeled “Medline Plus” (which is the
actual name of the free National Library of Medicine resource that was used, www.medlineplus.org), but in phase 3, most patients were able to find it and navigate to it after it
was relabeled “Health Topics.”
Almost none of the patients were able to grant proxy access to a family member. Usually,
this was because prior to meeting with us, they did not recognize this could be done.
When given a scenario about sharing data with a family member, many responded by offering
to share their username and password (which we classified as a “substituted task”).
In addition to this conceptual problem, patients also encountered difficulties from
the fact that the account enabled both obtaining access to another person's account as well as granting access to another person. Because both functions were unfamiliar to most patients,
patients frequently mistook one function for the other. These functions were labeled
as “Add Family Member” and “Health Record Sharing,” respectively.
The average System Usability Scale (SUS) score improved from 69.2 (SD: 20.5) among
the first group of participants to 81.9 (18.2) among the second group of participants
who saw the redesigned Web site (p = 0.049).
Themes
Five overall themes ([Table 2]) arose from the examination of task-technology fit and the analysis of the qualitative
data.
Table 2
Theme summary
|
Theme
|
Participant quotes
|
Identifier
|
|
Theme 1: Mismatch between user and technology
|
I don't know what Create a Medical Report means...I'm not familiar with that report
making process. I've never done that before
|
PID-01, 27 y. College educated
|
|
[Reads aloud] “Continuity of care.” Yeah, I'll have to learn about continuity of care
from someone, I think. I mean it just hits you foreign. You know, there is no continuity
of – to me, continuity is some kind of narrow, real narrow definition
|
PID-03, 74 y. College educated
|
|
[Reads aloud] “CCD export.” What does CCD mean? I don't understand. What is IPHR?
Why is everything so abbreviated over here? That's really annoying.
|
PID-04, 26 y. College educated
|
|
Theme 2: Mismatch between user and tasks
|
What's “Create a Medical Report”? What would you use that for?
|
PID-22, 53 y. College educated
|
|
All I would want to know is what are my laboratory results. I don't want to tell them
when I did it, where I did it, who ordered it. I wouldn't want to tell them all this
information. This looks like it's for a doctor.
|
PID-06, 31 y. College educated
|
|
Well, most people aren't going to know the name of the test they took. I mean a lot
of those tests are medical names. I would know the laboratory I went to, maybe the
date and time, but the name – the result name. value – I wouldn't know any of that
information
|
PID-13, 44 y. College educated
|
|
Theme 3: Seeking help
|
I don't know. I'm so bad. Is this a map to scroll down?
|
PID-11, 47 y. College educated
|
|
Could this be it? ... I'm sorry, folks… I'm not really sure this is it. So let me
just look at these categories again...no, I'm going to “Ask the Doctor for Help”
|
PID-19, 40 y. High school/GED
|
|
I don't know if I'm navigating very seamless on this
|
PID-22, 53 y. College educated
|
|
Theme 4: Using system to learn
|
Okay, discharge diagnosis. Okay, so at least, like, I can then go back and confirm
that, you know, whatever I was admitted for is the same thing I was discharged with
so there's no, like, billing error
|
PID-04, 26 y. College educated
|
|
Hospital Record. You see that? That's smart. So you can share with your dad right
there
|
PID-08, 18 y. High school/GED
|
|
This research help topics, it's great. It's great for the person who is newly diagnosed.
I wish I had this early on. That's a great one
|
PID-16, 41 y. College educated
|
|
Theme 5: Importance of storytelling
|
Request an appointment. Okay...And that would be really nice because the doctor I
see actually here, usually I have to call, usually the doctor's nurse is not available
when you call. Then they call you back. They do the appointment, but it's not like
very easy...so yeah, this would be really helpful, what I'd like to get the appointment
for a specific doctor
|
PID-17, 49 y. College educated
|
|
Well I'm just looking over the disorders, conditions, you know, the health issues.
And my son particularly has Crohn's disease, so I was just looking to see if I find
it here
|
PID-14, 46 y. High school/GED
|
|
...so I would like to know, like, you know, my mom's medical history. Like, I can't
remember everything about it, so you having that record and I can link into it so
whenever a physician then asks me what is your mother's – you know, do you have diabetes
in your family, I'm, like, I don't know, I can't think of it right now, I'll just
go into the portal and look up my mom. That'd be nice
|
PID-04, 26 y. College educated
|
Abbreviation: GED, General Educational Development.
Theme 1: Mismatch between User and Technology
Many of the patients, who had considerable experience with chronic conditions, displayed
fairly nuanced mental models of their (or their family member's) condition.[63]
[64] However, in navigating the patient portal, some had difficulty forming a clear understanding
of how the electronic portal worked. When there were mismatches between user mental
models and the technology, they manifested primarily as vocabulary misunderstandings,
as portal functionality that did not perform as the patient expected, and as requests
for clarification and help.
All 23 participants encountered problems with portal vocabulary on multiple occasions,
either misinterpreting terms or encountering terms that they did not understand at
all. Especially in the latter case, this led to frustration and self-doubt. A college-educated
participant said that she would need someone to help her interpret everything in the
portal: “I don't know. I don't know what this all means. You know, unless I'm sitting
there Googling every word, I wouldn't know what this means and it's not really fair
for me.” Most of these confusing terms were not medical but rather health information
management terms, such as “HIM” referring to health information management (and meaning
more specifically the hospital office that handles health information management requests),
and “CCD,” the HL-7 term that refers to the continuity of care document.
All of the participants also encountered situations in which the functions did not
perform as they expected. One example was that when patients clicked a button to create
a report for their doctor, many did not recognize that the report appeared as a separate
window (a PDF) in front and obscuring the view of the portal. Patients who did not
recognize that they were in a different window had great difficulty navigating back
to the remainder of the portal.
Theme 2: Mismatch between User and Tasks
Despite the patients' understanding of their medical conditions, most (19 out of 23)
did not have a very concrete understanding of health information management tasks.
For example, many patients had encountered the need to share their medical data, but
few were aware it was possible to export electronic copies of records or share their
medical record with family members or proxies. This sometimes led to mixed reactions
when the participant discovered portal features that did not align with their expectations
and/or beliefs.
Proxy access was a particular challenge. Most participants did not know that this
was possible, although, many when they learned about it agreed that it could be very
useful. Nevertheless, as described earlier, almost none of them were able to successfully
use the portal to grant proxy access to a (fictitious) family member. Furthermore,
not all participants welcomed the possibility of proxy access. One man said, semi-jokingly,
“Oh, wow. I don't know if I like that…. certainly not full access to my records, because
if myself and my brother were in line to get a ten-million-dollar inheritance, he
might want to see me dead, or I might want to see him dead.”
Some patients recognized the problems inherent in having medical records scattered
across multiple doctors, and were interested in consolidating data and sharing it.
Yet, the portal functions designed to help patients handle this problem did not appear
to be intuitive. For example, a portal function that allowed patients to input their
own data from other doctors was considered strange by almost all of the patients who
discovered it. One woman said that patient-entered laboratory data was unlikely to
be accurate, and was concerned about the workload involved. “All I would want to know
is what are my lab results. I don't want to tell them when I did it, where I did it,
who ordered it. I wouldn't want to tell them all this information. This looks like
it's for a doctor.” Another example was that many patients who agreed that they would
want to share their hospitalization data with another doctor outside the system did
not understand the concept of developing a report (using the CCD format). Instead,
many substituted a different task to accomplish the goal, such as taking notes or
screenshots of laboratory values.
Theme 3: Seeking help
The mismatches described under Theme 1 and Theme 2 meant that patients frequently
asked the interviewer for assistance understanding terms or tasks. In some cases,
the questions were needed because there was insufficient guidance on the portal. One
man said, “I'm not really sure this is it. So let me just look at these categories
again...no, I'm going to ‘Ask the Doctor for Help.’” Many participants who became
enthusiastic about the portal during the user testing asked to be helped through the
process of signing up for their own account.
However, others immediately asked for help without searching for instructions or attempting
the task. For example, when one woman was asked to do the proxy access task, she said,
“I don't know, because would I have to give them my password and login for them to
log in to see the information? Or is there some way I could send information to their
e-mail or something like that?”
Although some patients expressed frustration at the Web site (such as the woman cited
earlier who said, “it's not really fair for me”), many instead blamed themselves for
failing to understand or complete a task. For instance, one man took the responsibility
of not completing a task completely upon himself by saying, “I may have a problem.
I didn't look at this line.” Another woman said, “I mean, I feel like I'm letting
you down…” Others blamed both: “One thing I don't like is having to hit the Back button
all the time to get the prompts.... unless there's another way that I'm not seeing
to get to it.”
Theme 4: Using the System to Learn
As the participants explored the portal, they discovered and tested new functions.
In particular, most spent several minutes at the beginning of each session exploring
the menu options and clicking large buttons on the splash page to see what they would
do. One participant, looking at a shared record, realized what it was for, and said,
“So my son could be in it and I could be in it, too.” Another, viewing a list of previous
hospitalizations in the portal, noted, “And it gives you a list of all the hospital
records. So if I wanted to print my records, I could just go in here and get a copy
of them myself, right?” later adding, “Oh, I see. Okay, and it tells you all the dates
that you were admitted.” Generally, these discoveries led to positive reactions. One
person, discovering the “Health Topics” resource, said “If I was to type in ‘diabetes,’
I would expect information about it – which it does – and then all these various links
to it, which is very helpful.” After initial exploration, one person said approvingly,
“It's a good, useful access for a person at home or anywhere now. You're easily able
to go on your phone and log in and see what's going on.” There were, however, occasional
negative responses. One woman, who was interested to learn that she could add previous
laboratory tests to her portal record, said, with disappointment, “So I would have
to know the name of the test,” later adding, “not everyone knows the name of the test
they took.” Using the system to learn about the system can be considered a way of
improving user-technology fit by changing the user's own understanding.
Theme 5: Importance of Storytelling
Almost all of the participants (22 of 23) used personal anecdotes as a way of understanding
the portal's functions, particularly new functions that they were previously unfamiliar
with. For example, one woman with a child with chronic illness described a terrible
recent year in which her son had had to be hospitalized 11 different times, and talked
about how the portal account could be extremely useful for her to go back to look
at his laboratory data from those hospitalizations. Another, while reviewing the appointment
list in the portal, recounted how she had once confirmed multiple medical appointments
for her condition, only to receive an automated phone call saying that they were at
a different time. Another woman looking at the portal for the first time said that
creating a story would help her decide what to look at in the portal: “I'm just trying
to make a scenario while I'm at the hospital, what I would want to know.”
Discussion
Chronic illness imposes a significant workload on patients and caregivers, who need
to learn to manage physician appointments, disease education, and self-management
tasks, and even perform the “emotional work” of support and reassurance.[51]
[52] An important component of illness work is health information management, including
such tasks as tracking personal medical data and transferring medical information
from provider to provider.[49]
Therefore, in evaluating an electronic patient portal, we applied a task-technology
fit framework to improve not only usability but also to match the patient work.[54] The FITT framework identifies three important relationships to examine: the user-technology
fit, the user-task relationship, and the task-technology fit. In applying this framework,
we identified several areas in which a patient portal did not fully match the needs
of users, including the use of technical health information management terms and individual
functions that did not produce the results that patients expected. Remediating these
areas improved patients' ability to perform several common health information management
tasks with the portal.
However, our work on the user-task relationship also revealed that patients often
lack a clear understanding of the component of illness work that involves health information
management. For example, patients told us that their illness work required them to
share data from organization to organization or within their family. However, few
recognized that their medical record could be transformed into a shareable electronic
copy, or that their medical record account could be shared directly with family members.
Without this conceptual understanding of the task already in place, it was very difficult
for the portal to do a good job of supporting the user in accomplishing the task.
Portal developers may need to take responsibility for educating patients—not about
patient work, but rather about unfamiliar ways that portals can support patient work.
Generally, when a task-technology fit model is applied to a sociotechnical situation
(such as the use of an electronic health record [EHR]), there is an assumption that
the users agree upon and understand the dimensions of the task. This may not be true
in the case of patient work.
Our work suggests that an effective way to demonstrate the link between portal functionality
and patient work might be through narratives of fictional or archetypal patients.
We found that many patients, when learning about the portal, improved their own understanding
by linking portal functionalities to anecdotes from their own illness course or constructing
hypothetical narratives about fictitious patients. Developers of patient portals might
consider text or video narratives to help patients develop solid mental models of
portal functions. “Personas,” profiles of archetypal users accomplishing personally
relevant tasks, are used in user-centered design to help developers orient themselves
toward the needs of users.[65] Such personas could possibly also be transformed into helpful educational tools
for patients themselves.
User-centered product design involves understanding the needs, values, and abilities
of users to improve the quality of users' interactions with and perceptions of the
technology. In this study, we explored the views and needs of patients in iterative
phases toward the development of a patient portal. A significant conclusion of this
work is that incorporating the perspectives and needs of potential portal users will
allow the system to evolve in a more meaningful way for the target population. Developing
context-specific features based on user input led to improved acceptance and more
positive reactions from the users, which in turn has implications for mass adoption
and use.
Limitations
This study should be interpreted in light of several limitations. First, our study
sample was selected through convenience sampling and may not be representative of
larger target populations; specifically, patients who responded to our invitation
to test the portal already had familiarity with computers and felt comfortable using
them. This limits generalizability to less computer-literate populations. Second,
we included only individuals who themselves had chronic conditions or cared for family
members with chronic conditions; this was because previous work by ourselves and others
shows that portals are more frequently used by individuals with medical conditions.[28]
[66] However, this may limit applicability of our findings to healthier individuals.
Third, although we were able to recruit participants with low levels of numeracy,
our sample did not include individuals with very low levels of literacy, probably
because the computer literacy requirement ended up screening out very low-literacy
individuals. We also conducted all interviews in English, meaning that all participants
were comfortable communicating in English. These two elements are likely to limit
generalizability to the least literate populations as well as those with limited English
proficiency. Finally, although the sample size was determined by saturation in data
analysis, the possibility of larger samples leading to different conclusions should
not be overlooked.
We conducted this evaluation from the perspective of the FITT model, a validated model
showing that as these three dimensions of fit improve, technology adoption increases.[54] In the current study, we were able to show improvements in several of these fit
dimensions, but demonstrating a quantitative increase in user adoption was beyond
the scope of the study, which is an inherent limitation of the qualitative approach
we used.
Conclusion
Chronic illness imposes a significant workload on patients, and applying a task-technology
framework to evaluate a patient portal helped improve the portal's fit to patient
needs. However, it also revealed that patients often lack a clear understanding of
specific tasks that would help them accomplish PHIM work. Portal developers may need
to take responsibility for educating patients about patient work, in a way that developers
of clinical information systems do not need to do. In addition, patient understanding
of portals may be improved by providing narratives about hypothetical patients.
Clinical Relevance Statement
Clinical Relevance Statement
Electronic patient portals may fail to achieve their objectives of empowering patients
if they are not designed with patient needs in mind. We demonstrate that evaluating
a patient portal from a task-technology fit perspective can lead to substantive improvements
in patients' ability to accomplish health information management tasks.
Multiple Choice Question
A task-technology fit model demonstrates that, to improve the odds of technology adoption:
-
Improving usability is sufficient
-
Improving user training is sufficient
-
Improving task-technology fit is sufficient
-
Improving fit between users, tasks, and technology is sufficient
Correct Answer: The correct answer is option d. Task-technology fit models, such as those of Goodhue
and Ammenwerth, actually focus on the three-way fit between the person using the technology,
the task they are trying to accomplish, and the technology being used to accomplish
it. This perspective demonstrates that usability is only part of the solution. Historically,
TTF models have generally been used in professional or workplace settings, but in
the current project, we applied these concepts to patient work.
