Keywords
shared decision-making - decision support system - utility coefficients
Introduction
In a patient-centric clinical context, the care process is personalized according
to the characteristics and needs of the individual patient. This can be achieved through
shared decision making (SDM), a more and more increasingly adopted process in which
the physician and the patient reach a consensus in solving a decision problem, by
sharing information and values.[1] A key step in SDM is identifying patient preferences, so that they can be included
in a model that represents a generic decision problem, to make it specific to the
individual. Correctly incorporating patient preferences in SDM is fundamental, since
it is known that different subjects may evaluate the same health state differently.[2] The desirability of a health state can be quantified through a utility coefficient
(UC), a variable ranging from 0 to 1 that measures the quality of life (QOL) perceived
by a patient in relation to such condition. A strategy for solving a clinical decision
problem (i.e., selecting the optimal treatment for a specific patient) is comparing
the options in terms of quality-adjusted life years (QALYs).[3] For each option, to compute QALYs for a patient, his expected survival is split
in time intervals, each one (Ti
) spent in a specific condition Ci
. Assuming that Ci
is associated with a utility coefficient UC
i
, QALYs are computed as
While QALYs are mostly known as the outcome measure in cost-utility analysis (CUA),
where different health care programs are compared in terms of both economic affordability
and health outcomes for a target population, they may be used as well for individual
decision-making.[4] This article focuses on the latter, but in principle our tool may be used also for
CUA. Since, according to the different perspectives, either patients or healthy individuals
may be interviewed for UC elicitation,[5]
[6] in the following we will simply use “interviewee” to refer to a subject from whom
UCs are elicited, and “interviewer” to refer to the person leading the elicitation
process.
Eliciting reliable UCs from an interviewee can be challenging, and the interviewer
(ideally, a psychologist) faces two main difficulties. First, the utility theory provides
multiple methods for eliciting UCs,[2]
[7] among which the interviewer must select the most appropriate one. Second, a method
can be administered in different ways, and it is difficult, without guidance, to follow
the same procedure for all interviewees. Moreover, often the utility elicitation is
not led by psychologists due to lack of resources, and UCs are elicited during medical
appointments by physicians who are not familiar with the utility theory.
The utility theory provides multiple elicitation methods. Two well-known methods are
the time trade-off (TTO) and standard gamble (SG),[2] which work with hypothetical scenarios. To value a suboptimal health state S, in TTO, the interviewee must choose between living his expected lifetime (T1) in
the S state or to live shorter (T2 < T1) but in perfect health. T2 is varied until the
interviewee is indifferent between the two options and the UC is computed as T2/T1.
In SG, the interviewee must choose between living his expected life in the S state or accepting a gamble whose outcomes are complete healing or sudden death with
probability p. Again, p is varied until the interviewee is indifferent between the two choices, and the UC
is computed as (1–p). Thus, UCs quantify the loss (death risk in SG or life time in TTO) the interviewee
is willing to accept in exchange for complete healing. A third method, the rating
scale (RS),[7] requires the interviewee to rate S on a scale from 0 (= death) to 100 (= perfect health). Since RS is not based on the
loss/gain theoretical background, it produces values rather than actual UCs, but it
is simple to administer and is useful to rank multiple health states at the beginning
of an elicitation process. The literature suggests that elicitation methods are not
equally applicable to every interviewee, as they may be biased by specific characteristics
of both the interviewee and the evaluated health state. Besides the interviewee's
emotional status, the elicitation is influenced by the attitude of the interviewer,
who may adopt different strategies to interact with the subjects.[8]
In front of all the challenges described above, health informatics may provide instruments
to promote an unbiased and standardized elicitation. Over time, several computerized
tools have been proposed,[9]
[10]
[11]
[12]
[13]
[14]
[15] but they still present significant limitations. None of them support interviewers
in choosing the elicitation method; they implement a limited number of methods, most
of them are not integrated with models (e.g., decision trees [DTs][16]) that exploit the obtained UCs in solving a clinical decision problem. The few[9]
[13] that provide the latter functionality have been developed for a specific disease,
and cannot be applied to other clinical domains.
The work presented in this article aimed at enhancing one of such tools, namely UceWeb,[14] by designing and implementing new functionalities to support users in a comprehensive
shared decision analysis workflow.
Methods
Three innovative features are proposed (see the next three subsections). First of
all, since UceWeb included only TTO, SG, and RS, and since the first two methods may
be unsuitable for some interviewees, we extended the set of available elicitation
methods. Second, we formalized and implemented a decision support system (DSS) to
assist the interviewer in selecting the elicitation method that most suits the individual
interviewee. Finally, we integrated UceWeb with a software for running DTs.
To evaluate the obtained system, we performed a pilot study, whose methods are described
in the Evaluation section.
Enhanced Set of Elicitation Methods
SG and TTO show some applicability issues.[17] First, interviewees may act as zero-traders, i.e., they may refuse to trade/gamble (even hypothetically) their life-time in exchange
for perfect health, independently from their perception of the examined health state.[17] This may happen, for example, when an interviewee feels responsible for children
or other nonautonomous family members. In the following, interviewees having such
a feeling of responsibility will be referred to as “caregivers.” Please note that
this definition of “caregiver” does not mean that the interviewee is eliciting UCs
on someone else's behalf. UCs elicited from caregivers show a ceiling effect toward
1.[17] Another problem concerns elicitation for temporary health states (lasting less than
1 year), since the short duration may induce interviewees to be reluctant in the trading/gambling
task.[17]
[18]
[19] TTO is also unsuitable for valuing mild, not life-threatening, states,[17]
[20]
[21] since interviewees hardly trade life-time in this case. Thus, even when the interviewee
can distinguish between QOL of two different mild conditions, this distinction is
not captured by the elicited UCs. TTO results may also be affected by interviewees'
perception about life expectancy. In fact, TTO elicitation starts presenting the interviewee
with an estimate of his survival, as from national statistics, and a high difference
between this value and the interviewee's subjective expectation may bias the UC result.[22]
Thus, we integrated into our system a modification known as daily TTO (DTTO) that
should fit better for the above-mentioned cases.[19] Instead of considering the whole lifetime, DTTO assesses how much daily time an
interviewee would hypothetically trade in exchange for complete healing. According
toBuckingham et al,[19] the UC is computed as:
where the available daily time is computed as the difference between 24 hours and the interviewee's average sleep
duration. While solving the zero-traders problem, DTTO may be unsuitable for unemployed,
since they may be excessively willing to give up free time.
Finally, we integrated the willingness to pay (WTP) method,[23] which assesses the percentage of economic resources an interviewee would hypothetically
trade in exchange for complete healing. Like DTTO, WTP does not require to face the
potentially shocking concept of risk of death or reduced survival, thus being suitable
for risk-averse interviewees.
Decision Support for Selecting the Elicitation Method
To dispense the interviewer from being familiar with utility theory, we exploited
the above-described evidence to develop a DSS for the selection of the elicitation
method. The DSS knowledge base is composed of 12 rules ([Table 1]) describing the applicability of the different methods to specific circumstances.
Each rule is made up of four components:
Table 1
Rules implemented in the decision support system
|
Condition
|
Action
|
Target
|
Evidence level
|
|
1. Temporary health state
Nonchronic health state, lasting less than 1 year
|
A
|
TTO
|
1
|
|
S
|
DTTO
|
2
|
|
A
|
SG
|
3
|
|
2. Interviewee is unemployed
Interviewee who is not employed
|
A
|
DTTO
|
4
|
|
3. Mild health state
Health state that is not impairing, whose UC is expected to be close to 1
|
A
|
TTO
|
1
|
|
S
|
DTTO
|
3
|
|
4. Short life expectancy
The interviewee's life expectancy is lower than 5 years
|
A
|
TTO
|
2
|
|
5. Interviewee is a “caregiver”
The interviewee feels responsible for young children or nonautonomous adult family
members
|
A
|
TTO
|
3
|
|
A
|
SG
|
4
|
|
6. Health state involving risky decisions
Impairing health state potentially curable with highly risky operations (e.g., lung
transplant, heart transplant)
|
S
|
SG
|
4
|
|
7. Patient is zero-trader
Patient refuses to trade/gamble independently of his perception of the examined health
state
|
E
|
Any method
|
1
|
|
8. Patient's life expectancy expectation differs from the presented value
Patient whose life length expectations differ from the value he is presented with
during a TTO task.
|
A
|
TTO
|
3
|
Abbreviations: DTTO, daily time trade-off; SG, standard gamble; TTO, time trade-off;
UC, utility coefficient.
-
Condition: the interviewee's or the health state characteristics triggering the rule (e.g.,
“unemployed interviewee,” “temporary health state”);
-
Target: the elicitation method the rule refers to;
-
Action: the type of recommendation, i.e., “suggest” (S), “advise against” (A), and “exclude”
(E);
-
Evidence level: an integer ranging from 1 (maximum) to 5 (minimum), representing the supporting
evidence given by the literature for the suggested action.
For example, the first row in [Table 1] affirms that there is highly reliable evidence against using the TTO for assessing
the QOL of a temporary health condition.
Rules are interpreted by an inference engine whose workflow, implemented in Java,
is shown in [Fig. 1]. Following the interviewer-initiated DSS activation (A), the system calculates a
recommendation for each elicitation method (B). To compute recommendations, the engine
splits the triggered rules into rules suggesting the method, and rules advising against
it. For each subgroup the engine computes the best evidence level. In case the two
subgroups have the same evidence level, the number of rules with that level is used
to determine the subgroup to suggest. Recommendations are delivered graphically, exploiting
a traffic-light metaphor: green light if the method is suggested and red or yellow
light (depending on the reliability of the recommendation) if the use of the method
is not recommended. In [Supplementary Fig. S1] (online only) in the Supplementary Material we provide an example of recommendation
delivery, complemented with the complete traffic-light color legend ([Supplementary Fig. S2] [online only] in the same file). Leveraging on the received recommendations, the
interviewer selects the elicitation method (C). The system might deliver further advices
during the elicitation process. For example, during a TTO elicitation, if the interviewee
is presented with an estimate of life expectancy that differs from his expectation
(D), the interviewer receives a warning and can decide whether to proceed with TTO
(E). Testing the zero-trading condition is another key step in the flow (F) since
in this case the interviewer is recommended to proceed with another method. At the
end of the elicitation, both the interviewee and the interviewer can rate the elicitation
experience (G). The score thus collected will provide information on the interviewee's
attitude toward the method during future elicitations involving the same subject.
Fig. 1 DSS workflow: Workflow for guiding the choice of the elicitation method. DSS, decision
support system; U, utility coefficient.
Integration with Decision Trees
When DTs are used in SDM, they include all the health states the patient might experience
as a result of the available treatment options,[16] and every state must be assigned a UC. In this work, we integrated UceWeb with TreeAge
Pro,[24] a widely used software for formalizing and running DTs. The TreeAge Pro Object Interface
enables to open, update, and analyze DTs using programming languages (Java in our
case), thus integrating such functionalities in custom applications. Currently, the
UceWeb repository contain two DTs, developed within the EU project Mobiguide,[25] comparing different atrial fibrillation therapies, but it can be easily extended
by adding any model formalized with TreeAge Pro.
Evaluation
To evaluate the DSS and the overall usability of the system, we asked 51 volunteers
to perform elicitations using all the available methods. We selected three health
conditions that trigger the rules listed in [Table 1]. In particular, we included a mild disease (mild psoriasis), a painful but temporary
impairing condition (rib fracture), and a life-threatening disease potentially curable
with high-risk surgery (dilated cardiomyopathy). A fourth health state was “living
with an amputated leg,” which does not trigger any rule. Volunteers were enrolled
from high school students attending internships at our Engineering Faculty, master
students in Biomedical Engineering, and researchers working in our department. Interviewees
were asked to answer some socio-demographic questions (e.g., sex, age, caregiving
status, and occupation), the answers of which are summarized in [Supplementary Table S1] (online only) in the Supplementary Material. Caregivers were asked to examine leg
amputation, since we ideally aimed to isolate the effects of the caregiving status
on the obtained UC values. The other subjects were randomly assigned one of the four
selected health states. Health states were illustrated to interviewees using text
and infographics specifically designed for this study. As an example, in [Supplementary Fig. S3] (online only) of the Supplementary Material we provide an English version of the
description used to introduce dilated cardiomyopathy. Then UCs were elicited with
all the available methods. After the elicitation, the interviewees filled in the System
Usability Scale[26] (SUS) questionnaire, to rate the overall usability of UceWeb. The SUS score computed
from the questionnaire ranges from 0 to 100. Moreover, they rated each elicitation
method on a scale from 1 (I didn't understand the task/I was very uncomfortable doing it) to 5 (I clearly understood the task/I was very comfortable doing it).
Results
The overall result of our work is an integrated framework for SDM. Interviewers interested
in exploiting DTs for SDM can select a model in the repository. As a response, UceWeb
provides the list of the health states considered by the DT. For each state, the interviewer
can elicit a UC from the current interviewee. In this case, UceWeb activates the elicitation
process, and the interviewer may decide to use the DSS for selecting the best elicitation
method. The obtained, personalized, UC values are then set into the DT. In case it
is not possible to elicit one or more UCs, the tool anyway provides its user with
UC default values, representing population preferences taken from the literature and
used to quantify the DT during the model construction step. When all the states are
assigned a UC, UceWeb invokes TreeAge Pro methods to perform the analysis. A video
illustrating this functionality is available online (https://www.youtube.com/watch?v=_f3NLm8ayCE&feature=youtu.be). The following sections describe the results of the pilot study assessing the interviewees'
perception on the tool.
Elicitation Results
We excluded from the analysis those interviewees who did not provide socio-demographic
information. The final sample size was 48. For each respondent we collected a socio-demographic
profile, five UCs (one per method), and five rating scores (one per method).
First, we analyzed interviewees' willingness to engage in the hypothetical trade.
Methods involving the death concept (TTO and SG) showed a higher number of zero-traders
(11 and 6, respectively) compared with DTTO (0) and WTP (3). According to the literature,[17] this effect was more frequent when evaluating temporary (like rib fracture) and
mild (like mild psoriasis) health states ([Supplementary Table S2], online only). For such states, interviewees' reluctance to trade in TTO and SG
is reflected also by the resulting UCs ([Supplementary Table S3], online only): coefficients elicited through SG and TTO were significantly higher
than those obtained with other methods (Friedman test p-value < 0.005). Paired Wilcoxon signed-rank tests were performed to compare each
pair of elicitation methods. Test results regarding rib fracture suggest that the
median TTO value (0.98) was significantly higher than those of DTTO (0.78) and WTP
(0.94) (p-values of 0.034 and 0.043, respectively), and that SG (median value: 0.99) produced
significantly higher values than DTTO (p-value: 0.027). Test results for mild psoriasis indicate that all the median UCs from
a method involving the death concept (UCTTO = 0.79, UCSG = 0.8) are significantly higher than those from any other method (UCDTTO = 0.68, UCWTP = 0.61) (p-value < 0.05 for any pair comparison). Due to interviewees' reluctance to trade when
evaluating rib fracture and mild psoriasis, the distributions of the UCs elicited
through TTO and SG are skewed toward 1, with the ceiling effect being particularly
evident for rib fracture ([Supplementary Fig. S4], online only).
Usability Assessment
SUS scores ranged from 47.5 to 100 (interquartile range: 67.5-88.75; median: 78.75).
Since SUS scores above 68 are conventionally considered above average, we conclude
that interviewees considered UceWeb as a user-friendly application. Moreover, they
assigned positive rating scores to all the elicitation methods. Median rating scores
ranged from 4 (RS, SG, TTO, and DTTO) to 5 (WTP).
Despite the overall good usability, we highlighted some criticalities. The DTTO task
was difficult to understand. During our experiment, interviewees needed help to understand
the terms of the DTTO trading process, and struggled to interpret the proposed extra
sleeping time as an unfavorable tradeoff.[19] However, with the due explanations, interviewees seemed more confident with DTTO
than with TTO. Consequently, no interviewee was zero-trader in DTTO, DTTO UCs were
lower than TTO UCs, and no ceiling effect was detected.
Discussion and Conclusion
Discussion and Conclusion
The new version of UceWeb is an integrated environment for collecting and exploiting
QOL data in SDM. To our knowledge, UceWeb is the first tool that integrates functionalities
for eliciting UCs with both a UC repository and the facilities for running custom,
personalized DT models. Besides being useful for SDM, UceWeb may be used to collect
UCs within an international network. Since the elicited UC values are complemented
with the patient's (anonymous) profile, researchers interested in CUA can retrieve
from our repository specific sets of UCs that are appropriate for their target population.
Our set of elicitation methods is the widest among the tools known in the literature
and showed satisfactory usability during our study. UceWeb is also the first tool
including a DSS for selecting the elicitation method. The UCs collected in our experiment
confirm the rules implemented in the DSS. In particular, the temporary or mild nature
of the health condition led our interviewees to be reluctant to trade when using TTO
or SG, independently of the perceived desirability of the condition.
We are aware of the limitations of our study. First, we did not implement all the
variants of the SG and TTO methods. For example, a multistep variant of the TTO known
as “chained TTO” may be suitable to value temporary states, but in the literature
we found a lack of agreement on how to perform such elicitation.[27] We are also aware of the two variants of the TTO described by Artaso and Díez.[28] In the first one (“TTOqol”), interviewees trade QOL in exchange for life expectancy.
In the second one (“TTOequiv”), interviewees have to compare two health conditions
(A and B) by setting “time equivalence,” i.e., the amount of time spent in A that
is equivalent to a given amount of time in B, according to their perception. These
methods may be added in the future, after collecting enough information from the literature
to add rules on their applicability into the DSS knowledge base. In addition, it will
be also possible to explore the use of indirect methods that ask the patient to fill
in a health-related QOL questionnaire (e.g., Euroqol, https://euroqol.org/) and compute a UC value as a weighted sum of the given answers.[29]
Another limitation is that we only rely on DTs as formalism for clinical decision-making.
However, also influence diagrams could be used, and some literature suggests that
they may be more flexible in managing multiple time-distributed decisions.[30]
[31] Thus, we will evaluate the possibility to integrate UceWeb with tools for running
such models. An additional limitation is that TreeAge Pro is a commercial product
that requires a license. Currently, UceWeb is in a research-development phase. To
provide the same functionalities as a service, an agreement with the TreeAge Pro company
must be achieved.
Finally, as regard the interviewed sample, the low number of caregivers did not allow
us to draw definite conclusions about the impact of caregiving on the elicitation
process.
UceWeb is currently focused on the aspect of SDM that involves evaluating patient's
preferences related to a set of health conditions, with the goal of selecting the
decision option that maximizes a specific payoff, the QALYs. This is indeed just one
aspect of application of SDM, which has also been the focus of multiple-criteria decision
analysis techniques. These methods rely on the evaluation of patients' preferences
to rank the multiple criteria that have to be taken into account to compare different
decision options.[32]
[33] In the future, we could consider this as an interesting extension of UceWeb to provide
the possibility of a personalized selection of the payoff.
To test UceWeb further, we are planning additional experiments, in which we will enroll
more caregivers. In addition, we will plan a pilot study to collect feedback on our
tool also from interviewers, since this first test focused only on the interviewee's
perspective. We will also enroll clinical experts to assess how they perceive the
functionalities that are specifically dedicated to the medical decision process (e.g.,
the one for running DTs).
