Keywords
clinical outcomes - patient-reported outcomes - PROMs - methodology - validation
Introduction
In recent years, as health care has migrated to patient-centered care, there has been a drastic increase in the use of questionnaires designed to measure clinical outcomes reported by the patient.[1]
[2] These tools mainly enable the determination of the changes associated with therapeutic interventions, as well as the follow-up and prognosis of different pathologies, with the goal of performing the evaluation through a score. The results obtained through these questionnaires are also known as Patient-Reported Outcome Measures (PROMs).[3]
Specifically in the field of Orthopedics and Traumatology, PROMs have been fundamental to evaluate health aspects that are relevant to patients, their families and their support network.[3] Today, they are widely used in the clinical practice, especially in patients undergoing surgical interventions, enabling the collection of information on the impact on the functionality of activities of daily living, self-care, symptoms, and quality of life.[4] Moreover, they have an impact on scientific research, since they enable the comparison of treatments in a standardized way.[1]
These questionnaires can accurately measure the outcome of interest, provided they are used by the populations for which they were designed and tested.[5] In the event that these questionnaires are to be used in other populations, it is necessary to previously carry out a process of translation, cultural adaptation, and validation, which must be performed following a specific methodology. This workflow seeks to guarantee that the adapted questionnaire has the same characteristics as the original, in such a way that it is capable of correctly interpreting the results obtained in each use, avoiding errors of classification, diagnosis, or decision-making.[6]
In this context, the objective of the present methodological guide is to describe the process of translation, cross-cultural adaptation, and validation of questionnaires for clinical results in Orthopedics and Traumatology.
Translation, cross-cultural adaptation and validation of questionnaires
The translation, cross-cultural adaptation and validation of questionnaires should be carried out in those cases in which one wants to use a questionnaire that measures clinical results in a population with a language and/or culture different from those for which it was designed.
The first action is to contact the authors who originally designed the questionnaire and request permission to start the translation and validation process. The steps that must be followed, once the process is authorized, are described as follows:[4]
[7]
I. Translation and cross-cultural adaptation
It is important to bear in mind that a process of translation and cross-cultural adaptation will be necessary whenever the questionnaire is intended to be used in a country other than the country of origin, even in cases in which the language is the same or similar.[8]
The translation process is divided into four steps: 1. translation; 2. synthesis; 3. back translation; 4. committee of experts ([Table 1]).
Table 1
|
Step
|
Name
|
Task
|
Participants
|
|
1
|
Translation
|
Two translators, whose mother tongue is the original language of the questionnaire, must perform the translation from the original language to the target language separately. Thus, two different translations are are obtained
|
Translator 1
Translator 2
|
|
2
|
Synthesis
|
A consensus is reached between the two translations carried out, resolving the discrepancies
|
Translator 1
Translator 2
|
|
3
|
Back translation
|
Two translators, whose mother tongue is the target language of the questionnaire, must perform the translation to the original language based on the consensus version separately. Thus, two different translations are obtained
|
Translator 1
Translator 2
|
|
4
|
Committee of experts
|
The translations are reviewed and it is observed that the back translation is similar to the original questionnaire. Discrepancies from the consensus version are reviewed, and a preliminary version is produced
|
Translators[2]
[3]
[4]
Methodologist
Linguist
Research team
Experts in the field
Others
|
The initial translation must be performed based on the questionnaire in its original language by at least two independent translators who must be native in the original language. Then, a consensual version, which is called a synthesis, is generated based on the two translations. Next, in the back-translation step, the synthesis is translated back to the original language of the questionnaire, to check that there are no relevant discrepancies regarding the original tool. After this review, the synthesis version is reviewed by an expert committee made up of the translators, a methodologist, a linguist, the research team, and other people who can contribute to the review, such as community representatives ([Figure 1]). And so, we reach what we will call the preliminary version of the questionnaire.
Fig. 1 Members of the expert committee for the pre-final version of the questionnaire.
The preliminary version should be evaluated by 30 to 40 patients.[4] Although some studies show that this process can be carried out in any population,[8] other authors[4]
[7] point out that it is recommended that it should be carried out by patients who have the pathology or painful syndrome evaluated by the questionnaire, since this enables a better approach and generates better feedback for the next steps.
Special attention must be paid when obtaining a representative population in terms of their level of schooling. According to the 2016-2017 National Health Survey held by the Chilean Ministry of Health, 77% of the Chilean population has less than 12 years of schooling, and 23.7%, less than 8 years, so the target population should be a representative sample of that population.[9]
The questionnaires must be evaluated respecting the properties with which they were created. In the case of most questionnaires for clinical results in Orthopedics and Traumatology that collect PROMs, they are designed to be self-administered by patients, so, at this stage of the evaluation, it is important that they fill out the questionnaire themselves, with instructions to answer all of the questions. On the other hand, if the questionnaire has been designed to be administered by a third party, it is most appropriate that all the questionnaires be administered by the same person.[10]
After the application of the preliminary questionnaire, a semi-structured interview is carried out, in which the patient is invited to comment globally on what he or she thought of the questionnaire, and is asked about difficulties or conflicts when trying to answer any item in detail. The meaning attributed by the patient to each question should be evaluated to ensure that the final version maintains its equivalence to the original version. This interview can be recorded, so all responses can be tabulated later in order to identify those items that generated conflict in more than 15% of the respondents.[6]
Finally, a new meeting of the expert committee is planned with the aim of discussing these questions and making the necessary modifications to develop the final version ([Figure 2]).
Fig. 2 Process for obtaining the final version of the questionnaire.
Undoubtedly, this process, which requires great effort, is necessary to be able to produce a version that is similar to the original version. Following these guidelines brings us closer to having a tool that can be used reliably in a population. However, despite the fact that up to this point we have obtained very useful information when comprehending the process of understanding the questionnaire, this does not provide information on validity or reliability.[11]
[12]
II. Evaluation of psychometric properties
The following steps must also comply with a rigorous process, in which the psychometric properties of the questionnaire will be sought, which will consist in the evaluation of validity and reliability.[11]
[13]
[14] In this process, the questionnaire in its final version must be used by a representative sample of the population at whom it is aimed ([Table 2]).[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
Table 2
|
Item
|
Description
|
|
Subjects
|
Representative sample of the population in whom I intend to use this tool.
Sample size of 5 to 10 subjects for each question contained in the questionnaire[15]
[16]
|
|
Example 1
|
The Arthritis Research UK Musculokeletal Health Questionnaire (MSK-HQ) is a recently developed PROM that assesses quality of life related to musculoskeletal health. In its creation, it was validated with a sample of 570 patients with osteoarthritis. In 2019, in the same country, a study was conducted to evaluate the validity and reliability of the MSK-HQ in people with inflammatory arthritis.[17]
|
|
Example 2
|
The Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) is a PROM that is widely used to assess pain, stiffness, and function in patients with osteoarthritis of the hip and knee.[18] Although the WOMAC has also been used to assess postoperative foot and ankle surgery, it was not validated for this purpose. Ponkilainen et al. carried out the validation process in 2019.[19]
|
|
Instruments and benchmark tests
|
Use the tool to be validated together with the benchmark tests, which may be a similar questionnaire, a clinical examination, a laboratory examination, imaging etc.
|
|
Example 1
|
In a questionnaire created and validated by Schnetzke in 2016, the objective was to evaluate the range of motion of the patient's wrist and elbow. The self-administered tool contained questions about the ability to perform certain movements, and the goniometric evaluation of a trained examiner was used as a reference test for the validation process.[20]
|
|
Example 2
|
Pardis et al. performed the validation of the Athlete Disability Index (ADI), a questionnaire to assess disability associated with low back pain in athletes. For the process, they used this questionnaire in conjunction with the Oswestry Disability Index (ODI), and the Roland-Morris Disability Questionnaire (RDQ), both tools originally used to evaluate low back pain.[21]
|
|
Reevaluation
|
The application of the evaluations must be repeated in a certain time
|
|
Example 1
|
In Spain, the SEROD group carried out the validation of the Hip and knee questionnaire, which assesses the impact of musculoskeletal pathologies on the quality of life of patients. To do this, they included patients who underwent total knee arthroplasty and used the questionnaire together with the benchmark test, before surgery and 6 months after surgery.[22]
|
|
Example 2
|
In Korea, the Core Outcome Measures Index questionnaire was validated in patients with degenerative lumbar pathology. The questionnaire was used by a group of patients in their first consultation, in conjunction with the reference test, and after 2 weeks it was used again in conjunction with a transition question (no changes, slight changes, moderate changes, many changes). This correlated with changes in the questionnaire.[23]
|
Validity
The validity of a questionnaire corresponds to the ability to adequately measure what it wants to measure, and correctly evaluate the characteristic for which it was created.[6] Three types of validity are defined:
-
Content validity: it refers to the extent to which the items in the questionnaire are representative of the characteristic that you want to measure or for which it is designed. This process is crucial in the stage of development of the questionnaire; however, it is also crucial at the time of validation. A panel of experts in the clinical outcome to be measured has the task of evaluating the validity of the content.[12]
[24]
[25]
-
Construct validity: it assesses the degree to which the questionnaire reflects the characteristic or concept that it wants to measure,[26] estimating its association with other variables (or measures of a construct) with which it should have a positive, negative or null correlation. It is worth mentioning that the construct is defined as the “underlying theory in the phenomenon or concept to be measured. It is an unobservable quality in a population of subjects”.[6]
-
Criterion validity: it corresponds to the relationship of the score of each subject with a gold standard that measures the same characteristic.[6]
[8] This validity is made up of two dimensions:
-
Concurrent or convergent validity: the degree to which the result of the questionnaire agrees with some standard at a given moment in time; and
-
Predictive validity: the degree to which it is able to predict a certain result.
Reliability
The reliability of a questionnaire is the consistency of its results, which can be evaluated using internal consistency, intraobserver reliability, and interobserver reliability[12]:
-
Internal consistency: it reflects the degree to which the items in the questionnaire are correlated, or if they are consistent in measuring the same phenomenon. Internal consistency is commonly estimated using the alpha coefficient, also known as Cronbach alpha.[27] The Cronbach alpha ranges from 0 to 1, and a number close to 0 indicates that there is no internal consistency, or that the items are not correlated. A number close to 1 indicates perfect internal consistency. A cut-off point for adequate consistency has been stated to be 0.7.[28] (Appendix 1)
-
Intraobserver reliability: it corresponds to a way of measuring the stability of the questionnaire scores, in the same subjects and with the same method, at different times.[6]
[12]
[14] This will enable the evaluattion of the level of agreement in the responses of the subject at two or more intervals of time ([Figure 3]). This process can be carried out in two ways:
-
With an interval of one to three weeks from the first application, expecting that there is no significant change in the responses, because no significant changes in the patient's condition are expected either; and
-
According to a clinical milestone in which a result is expected. For example, using the questionnaire before and after a surgical intervention through which improvement is expected.
-
Interobserver reliability: this property assesses the concordance or agreement between 2 evaluators who apply the same tool to the same subject,[6] in the event that the questionnaire is not self-administered.
Fig. 3 Steps to obtain the psychometric properties of the questionnaire.
It is important to mention that not all the items of validity and reliability correspond to the process of all the questionnaires in Orthopedics and Traumatology. It is paramount to evaluate, in each case, the steps that are required for the correct development of this stage.
Conclusions
In recent years, PROMs have made it possible to improve the quality of patient care,[29]
[30] especially in the field of Orthopedics and Traumatology, due to the special attention dedicated to results related to pain, functionality and quality of life.[10]
Translation, cross-cultural adaptation, and validation can be long, difficult or expensive processes; however, they are essential when using a PROM in the clinical practice. This process is necessary even when one wishes to apply a questionnaire in countries with the same language.[6] Sometimes it is assumed that cultural adaptation to a different language guarantees the psychometric properties of the questionnaire, which can lead to errors in evaluations. It is necessary to complete the validation process rigorously, measuring the validity and reliability of the measurement tool.
Given the importance of the use of PROMs adapted and validated in the clinical practice to improve the control and follow-up of patients, the Organisation for Economic Co-operation and Development (OECD) has launched an initiative for the systematic collection of PROMs.[31] In the field of Orthopedics and Traumatology, this initiative promotes the evaluation of patients undergoing elective hip and knee arthroplasty prior to surgery, and 6 and 12 months after surgery.[32] The creation of a network for the systematic collection of PROMs in all countries will help to investigate the determinants of quality in medical care, to make national and international comparisons, and to align the practice with health policies.[31]
Therefore, it is necessary to promote the use of PROMs in the different health problems associated with the specialty of Orthopedics and Traumatology, using tools that have been translated, adapted and validated to the corresponding population, in order to make their use part of the routine of the clinical practice and thus access a common and objective language. (Appendix 2)
