Key words survey - accreditation - quality of care - mammary carcinoma - breastcancer
Schlüsselwörter Befragung - Zertifizierung - Versorgungsqualität - Mammakarzinom - Brustkrebs
Introduction
Patient surveys are an established tool for quality control in healthcare organizations. Despite criticisms that collecting data on patient satisfaction produces data with a strong ceiling effect, is highly subjective and strongly dependent on non-quality relevant factors (e.g. hotel-type services), there are many arguments in favor of patient surveys. Patient surveys which collect patient experiences in the form of more or less objective reports of events as described by the patients themselves (e.g., on the information available to patients and the organization of care) can offer insights into areas which are not accessible using other sources of information [1 ]. Patient surveys can look at the human aspect in care (communication, privacy, need for information, etc.) and the perceived efficacy of care [2 ], [3 ]. The reports on events are affected far less by bias, subjectivity and expectations than surveys which only focus on patient satisfaction. Numerous publications have demonstrated the correlation between patient experiences (including patient satisfaction) and outcomes, e.g. compliance with guidelines, mortality, re-admission rates, patient safety, clinical efficacy, or compliance with treatment [4 ]. The results of surveys of care procedures and the context of care (e.g. the communication skills of staff) can be used for the organizational development of healthcare organizations by analyzing the problems and taking the appropriate steps to improve [5 ].
Accreditation of Breast Centers in the German federal state of North Rhine-Westphalia (NRW) by the Ministry of Health of North Rhine-Westphalia was introduced in 2003. Breast Centers in NRW can request accreditation by the accreditation body of the medical association of Westphalia-Lippe (Äkzert) which examines and documents their compliance with the requirements issued by the state of NRW. The number of Breast Centers in NRW fluctuates slightly, but NRW usually has around 50 accredited Breast Centers. Since 2005, the list of accreditation requirements includes an annual survey of breast cancer patients using a standardized questionnaire and a standardized survey design [6 ]
[
1
]. As part of its (re-)accreditation audit Äkzert also looks at the findings of the patient survey and the measures for improvement based on the patient survey. The concept of the survey was developed together with the Center for Health Services Research Cologne (ZVFK) as part of the “Concerted Action against Breast Cancer” of the Ministry of Health of NRW [7 ]. Patient surveys were already being carried out in many healthcare organizations prior to 2005, but they were neither standardized nor validated and could therefore not be used to compare healthcare organizations. Moreover, these often unsystematic surveys rarely permitted developments to be monitored over time.
Since 2006, the ZVFK and the Institute for Medical Sociology, Health Services Research and Rehabilitation Science (IMVR) of the Faculty of Human Sciences and the Faculty of Medicine of the University of Cologne have been responsible for carrying out the patient surveys in the Breast Centers in NRW. The patient surveys are done to allow a formative evaluation of the Breast Cancer Centers and to monitor the quality of the care provided by the Centers. Feedback of the findings is given in the form of comparisons between facilities (benchmarking) and developments over time. The aim is to uncover any deficits in care and to promote mutual learning to encourage improvement.
This article presents the questionnaire along with selected results obtained over the course of 10 years and describes how feedback is given to the Breast Centers. As the survey design and the processing of the survey have already been described elsewhere [8 ], they will only be dealt with summarily here.
Material and Method
Design and processing of the survey
Patients who meet the following criteria are included in the survey:
Undergoing surgery in a surgery unit of an NRW Breast Center during their current stay in hospital
Discharged during the period of the survey, i.e., between 1st February to 31st July
At least one finding of malignancy
At least one postoperative histological finding of breast cancer
ICD code of the confirmed diagnosis is C50.x or D05.x
Primary diagnosis, i.e. not local recurrence
The survey is a complete survey carried out over a defined period of time (from 1st July to 31st July every year) and is done by mail around one week after the patient has been discharged from hospital. The hospital staff is responsible for asking patients to participate in the survey and asking them to sign the consent form. The Breast Center is also responsible for sending the patientsʼ consent forms back to the IMVR. If a patient consents to participate, the staff will forward the patientʼs clinical data to the IMVR. The IMVR then sends the questionnaire together with a prepaid return envelope addressed to the IMVR to the patientʼs home address. In accordance with the Total Design method [9 ] two further attempts are made to contact the patient in writing. Teleform software is used to scan the questionnaires after they have been returned to the IMVR, and the scanned data is checked for errors.
The Cologne Patient Questionnaire for Breast Cancer (CPQ-BC)
The Cologne Patient Questionnaire for Breast Cancer (CPQ-BC) was developed by the IMVR[
2
] in collaboration with the Quality Development Working Group, an association of all institutions involved in the Concerted Action against Breast Cancer in NRW. Pretesting of the questionnaire was done in 2005 using 310 surveyed patients, and the results of this pretesting were used to revise the questionnaire before the final version was approved. The CPQ-BC was based on the already established and validated Cologne Patient Questionnaire (CPQ) [10 ], which was expanded for use in Breast Centers by the inclusion of specific breast cancer and center-specific aspects [11 ], [12 ]. The questionnaire consists of several modules. In addition to questions on social and demographic factors and subjective health outcomes such as quality of life the questionnaire includes numerous scales which measure different aspects of care in the Breast Center. These aspects are assigned to different modules such as hospital organization, hospital services, interaction with staff, psychosocial information needs, involvement in treatment, discharge from hospital, and overall assessment of the patientʼs stay in hospital. The scales usually consist of at least three closed questions (items) with scale-specific response categories. Quality of life was initially assessed using the generic SF-36 questionnaire [13 ]. The SF-36 questionnaire was later replaced by the EORTC, a cancer-specific tool to assess health-related quality of life [14 ]. The clinical information provided by the hospital staff offers additional information on disease-specific and treatment-specific patient characteristics such UICC staging, grading, type of procedure and ASA classification.
The Cologne Patient Questionnaire for Breast Cancer 2.0 (CPQ-BC 2.0)
In 2012 the IMVR, working in cooperation with the accreditation body Äkzert, decided to revise the questionnaire. One of the principle aims of the revision was to collect data with greater practical relevance which would make it easier to develop measures for improvement. Another important aim was to improve the measurement of differences in patientsʼ assessments of hospitals. The goal was primarily to reduce so-called “ceiling effects”, i.e. performance indicators to which the overwhelming majority of persons questioned give extremely positive replies. Ceiling effects are a common phenomenon in patient evaluations and cannot be completely eliminated as they are simultaneously an indication of the high standards of care provided by Breast Centers. Various standard techniques used for the compilation of questionnaires were employed to achieve these aims:
Use of “questions about events”, based – as much as possible – on accreditation requirements;
Rewording of items and/or adaptation of response categories and scoring;
Avoiding scoring items according to the level of the patientʼs agreement or disagreement; instead, use of question-specific response categories where possible.
In addition, certain contents were replaced by new contents with a greater practical relevance, e.g., questions on the support available to relatives or the help given to prepare patients for being discharged from hospital. Certain tried-and-tested items and the assessment of quality of life were retained unchanged. The revised questionnaire was developed by a working group of the IMVR, based on the current state of knowledge on the development of questionnaires, and was subjected to an internal review. Subsequent to the internal review, cognitive pretest interviews using the Think Aloud method [15 ] were carried out with a total of 12 breast cancer patients to test the comprehensibility, practical viability and completeness of the questionnaire. The changes considered necessary after the interviews were successively implemented during the interview stage until all items were considered suitable. The CPQ-BC 2.0 was first used in the patient survey carried out in 2014. The scales used in the new questionnaire are shown in [Table 1 ]; they categorized according to various aspects of the patientʼs stay in hospital. The Cronbachʼs alphas for 2014 and 2015 shown in the table measure the internal consistency of the scale, i.e. the extent to which the individual items in a scale correlate with each other. Cronbachʼs alpha ranges between 0 and 1, but it should not drop below 0.70 if possible. In addition to the scales, many aspects of the patientʼs stay in hospital were assessed with the help of meaningful individual items which have not been included here for reasons of space.
Table 1 Overview of scales and Cronbachʼs alpha in 2014 and 2015, years in which the CPQ-BC 2.0 questionnaire was used.
Scale
Content
α 2014
α 2015
Hospital organization
Process organization
Measures waiting times and difficulties in arranging consultations
0.82
0.82
Hospital services
Room facilities
Measures room facilities
0.83
0.83
Customer-friendly infrastructure
Measures facilities and services available outside the patientʼs hospital room (e.g. shopping facilities, receiving visitors)
0.72
0.71
Cleanliness
Measures cleanliness
0.76
0.79
Quality of the patientʼs interaction with physicians
Empathy
Measures the physiciansʼ empathy
0.94
0.94
Confidence in physicians
Measures the patientʼs confidence in the physicians
0.92
0.91
Support by physicians
Measures the perceived support given by physicians
0.92
0.91
Quality of the patientʼs interaction with nursing staff
Confidence in nursing staff
Measures the patientʼs confidence in the nursing staff
0.95
0.94
Support by nursing staff
Measures the perceived support given by nursing staff
0.93
0.93
Psychosocial information needs
Health-related behavior
Measures the unfulfilled need for information with regard to health-related behavior
0.87
0.86
Work
Measures the unfulfilled need for information with regard to continuing work
0.88
0.86
Assistance in daily life
Measures the unfulfilled need for information with regard to the assistance available for activities of daily life
0.72
0.74
Disease
Measures the unfulfilled need for information on the disease
0.87
0.82
Treatment
Measures the unfulfilled need for information about the treatment
0.79
0.76
Involvement in treatment (shared decision-making)
Physician facilitation of patient involvement
Measures the perceived involvement of the patient in decision-making
0.84
0.85
Patient participation in decision making
Measures the patientʼs participation in decision-making
0.87
0.87
Discharge
Preparation for discharge
Measures extent of preparation for the time after the patient has been discharged from the Breast Center
0.84
0.84
Involvement in the planning of further treatment
Measures the involvement of patients and their next of kin in planning further treatment after being discharged
0.89
0.90
Use of medication
Measures the information given to the patient on the appropriate use of medication after discharge
0.81
0.82
Overall stay in hospital
Empowerment of patients
Measures the extent to which patients are empowered by the care in the Breast Center to understand their disease and cope with it
0.88
0.88
Feedback given to Breast Centers
Every participating surgical unit receives both a digital and a printed copy of the results of the survey by the end of October of the same year in which the survey was carried out. Results are presented in the form of a general report on all surgical units as well as an individual evaluation of all scales and items for the respective surgical unit. The general report provides the results across all surgical units differentiated according to surgical unit and presented in the form of anonymized benchmark graphs. The benchmark graphs (see [Fig. 1 ] for one example) allow every individual surgical unit to compare its own results with the results of other sites. Every surgical unit is identified by a hospital number; every surgical unit is only given its own number. The findings report also includes comparisons between the mean values of scales compared to previous years for all Breast Centers. This is done to show general trends. To simplify the interpretation of the scales, a scoring system from 0–100 is used to transform the scores of all scales. This creates a standardized way of reporting the results of different scales, even if the scales initially used different response scoring systems.
Fig. 1 Depiction of results using the benchmarking system: patient responses for every surgical unit (hospital) to the question of participation in clinical trials.
The additional evaluation provided to each individual surgical unit includes
a ranking of all surgical units,
a comparison of the mean values of scales for the different surgical units of a Breast Center and comparisons across different surveyed years, and
an evaluation of the frequency of responses of the individual items for patients treated in the individual surgical units.
The ranking shows the unitʼs overall performance in the survey; to create the ranking, the means of all the scales shown in [Table 1 ] are added and divided by the number of scales. The ranking also references the individual unitʼs performance with regard to the individual scales evaluated in the survey. The system used for ranking was also revised in 2014 to take account of the fact that there are often only minimal differences in the mean values of scales between the individual ranks. Because of this, since 2014 the results do not report each individual unitʼs ranking; instead, surgical units are grouped into quintiles. The best surgical units are grouped into the first quintile, and hospitals with the poorest results are grouped into the 5th quintile. Each surgical unitʼs overall ranking across all scales is now shown as a quintile (see, for example, [Fig. 2 ]). A published analysis into whether the survey results depend on the case-mix of individual hospitals showed that patient populations in Breast Centers are highly homogeneous and that the benefits of risk adjustment are low [16 ]. For this reason, rankings are not adjusted for risk. However, comparisons between the different units showed that the numbers of persons surveyed for every unit differed widely, a finding which was highlighted several times in the feedback report.
Fig. 2 Ranking of an individual surgical unit with regard to specific individual scales and overall; results presented as quintiles.
Benchmark workshops
In addition to the written feedback, every year after receiving the findings of the survey, representatives from the different Breast Centers are invited to participate in a half-day benchmark workshop. The most important results of the most recent survey are presented at the workshop. The aim is to promote the exchange of experiences between Breast Centers. After the initial presentation of results, participants split into different groups to attend one of three moderated workshops held simultaneously which focus on specific topics. The topics are based on the findings of the most recent survey. In the workshops, the staff and management of different Breast Centers exchange information on successful and unsuccessful practices in patient care. The aim is that units with a poorer ranking will learn from hospitals with the best ranking. After the workshops the key results and the specific measures for improvement are presented in a plenary session. An evaluation of the benchmark workshop held in 2009 showed a positive correlation between the participation of Centers in the benchmark workshop (the topic was “improvements in nursing care, particularly through the use of Breast Care nurses”) and the survey outcomes measuring the topics discussed in the workshop (perceived support given by nursing staff) the following year [17 ].
Selected Results for the Period 2006–2015
Selected Results for the Period 2006–2015
Survey response rate and representativeness
Around 40 000 patients returned their completed questionnaires in the survey period between 2006 and 2015. Over the course of the survey period, the number of surveyed persons increased year on year ([Fig. 3 ]). The percentage of patients who gave their consent to participate in the survey during their stay in hospital dropped from 90.8 % in 2006 to 83.5 % in 2015. The response rate for returned questionnaires is very high when measured against the number of patients who consented to participate in the survey, with the response rates ranging between 88.6 and 85.6 %, although the rates have decreased slightly over time [9 ]. According to the NRW Cancer Registry, a total of 16 972 women and men developed primary breast cancer in NRW in 2013 [18 ]. The percentage of persons who were diagnosed and surveyed in the same year (n = 4146) was therefore 24.4 %. The number of Breast Centers participating in the survey varies, ranging between 49 and 52 Breast Centers and 83 and 95 surgical units in NRW, depending on the survey year ([Fig. 3 ]). Not all of the surgical units which were part of an accredited Breast Center in 2006 are still accredited. Some of them are no longer accredited because of the limited number of cases treated in the unit. In general, however, these units continue to participate in the patient survey. As at the same time a number of newly accredited units have been included in the survey, the most recent survey reported a ratio of 77 accredited to 8 non-accredited surgical units in NRW. All of the 52 Breast Centers accredited by Äkzert participated in the patient survey.
Fig. 3 The numbers of participants and response rates for the patient surveys in the period 2006–2015.
The revision of the questionnaire was found to be beneficial with respect to the internal consistency of the scoring. The scores for the CPQ-BC 2.0 in the surveyed years 2014 and 2015 were between 0.71 and 0.95, confirming the high level of reliability of the revised or new scales.
A comparison of samples of the 2010 patient survey with regard to age, gender, T-stage and ASA classification with national data on patients with primary breast cancer compiled by the Robert Koch Institute [19 ] and the AQUA quality report [20 ] showed that surveyed patients in NRW were slightly younger with an average age of 60 years, that there were slightly less male patients with breast cancer (0.4 %), and that the patients in NRW were somewhat healthier based on their ASA classification compared to the overall national averages for patients in Germany.
Psychosocial information requirements
The CPQ-BC also includes questions about unfulfilled psychosocial information needs, based on the CAPIN scale (“cancer patient information needs”) developed by Neumann et al. [21 ]. A selection of the questions, which have largely remained the same over the years, is presented below ([Fig. 4 ]). The wording of the question is “Would you have liked more information from the Breast Center on the following topics?” and every item can be answered either “Yes” or “No”. [Fig. 4 ] shows the percentage of surveyed persons who answered the questions about different areas of information with “Yes” and thus have a subjectively unfulfilled need for more information. Values for the items “Measures to promote health” und “Help in daily life” are only given up to the year 2013, because from 2014 the items used in the CPQ-BC 2.0 changed, and the results were therefore not directly comparable.
Fig. 4 Psychosocial information requirements: changes in selected items over time (from 2006 to 2015); percentage of surveyed patients who answered the question “Would you have liked more information from the Breast Center on …” with “Yes”.
The reported need of patients is greatest for additional information on actions which can be taken in addition to therapy (unfulfilled need: 35–44 %), while the need for more information on psychosocial support is significantly lower (unfulfilled need for more information: 11–23 %). The need for more information decreased significantly over time for all items. Nevertheless, in certain areas, up to one third of patients still reported unfulfilled needs for more information [22 ], [23 ], [24 ].
Information on self-help
The list of requirements for accreditation explicitly stipulates that each Breast Center should provide its patients with information material and the contact details of self-help groups [6 ]. According to the information provided by the patients themselves in the survey, around two thirds of patients reported that they had received information about self-help groups ([Fig. 5 ]). The percentage of patients who reported that they had received information increased over time, rising from 62.6 % in 2006 to 70.0 % in 2015 [25 ]. Nevertheless, in 2015 more than one quarter of patients were still not receiving information on self-help groups [26 ].
Fig. 5 Information on self-help: percentage of surveyed patients who answered the question “Were you given information on self-help groups in hospital?” with “Yes”; Period from 2006 to 2015.
Discussion
Over the 10 years of patient surveys carried out by the IMVR in the Breast Centers accredited by the medical association of Westphalia-Lippe, data have been collected from more than 40 000 patients with primary breast cancer treated in one of the NRW Breast Centers between 2006 and 2015.
Questionnaires and response rate
The number of patients surveyed between 2006 and 2015 has increased year on year, primarily because the numbers of patients who underwent surgery following the early detection and treatment of very small tumors detected during mammography screening has also increased ([Fig. 3 ]). The slight decrease in the percentage of patients who gave their consent to participating in the study during their stay in hospital could be due to the fact that processes in hospitals have become shorter and patients spend less time in hospital; another reason could be that patients now receive more requests to take part in surveys and clinical trials. The response rate of between 85.6 and 88.6 % is extremely high, although it has decreased slightly over time – which corresponds to the slightly decreased willingness to participate described above. A high response rate is typical for breast cancer patients; in this case it is reinforced by the approach used for the survey which is based on the Total Design method [9 ], with three attempts made to contact the patient and the provision of prepaid return envelopes. A comparison of data from our 2010 patient survey with the national data showed that our data were largely representative, with only slight differences.
Psychosocial information requirements
Unfulfilled patient needs for more information were investigated using selected findings on psychosocial information. The significant decrease in patient need for more information highlights the efforts undertaken by the Breast Centers to provide their patients with the necessary information. Nevertheless, even in recent years up to one third of patients still had unfulfilled needs for more information about certain topics. Because breast cancer patients now spend less time in hospital [27 ] and the care given in Breast Centers has intensified, it is worth considering whether it is useful and feasible to provide breast cancer patients with information about the different aspects of disease, the associated challenges and the available support while they are still in hospital. This question and similar issues regarding breast cancer patientʼs information needs are being investigated in the PIAT study which is also being carried out by the IMVR [22 ], [23 ], [24 ]. The question is whether the physicians and therapists who treat the patients after they have been discharged from the Breast Center or e-Health technologies [28 ] would be a suitable means of providing information at the appropriate time to patients according to patientsʼ individual needs. Although there have been a few scattered longitudinal studies, there have been no routine, comprehensive studies in Germany on the information offered to breast cancer patients after their discharge from hospital.
Information on self-help
The rise in the percentage of patients who know about self-help indicates that Breast Centers have been implementing the required changes, which is particularly impressive given the ever shorter stays in hospital [25 ]. But as in 2015 around one quarter of patients still had not received information, there is still room for improvement. A detailed comparison of the information on self-help groups in the Breast Centers of NRW with data from 2007 shows that information on self-help varies between patients and even between hospitals [26 ]. Younger patients and patients with higher levels of education receive information on self-help significantly more often. An analysis based on the evaluation of the 82 surgical units included in the study found, after controlling for case volume, that patients treated in teaching hospitals were informed significantly less often about self-help than patients treated in hospitals which were not training hospitals. Training hospitals may be finding it difficult to offer outstanding patient care and provide excellent training while simultaneously involving outside partners such as self-help groups in the care of their patients.
Strengths and weaknesses
The strengths of this survey compared to other patient surveys are the extremely high rate of response, the use of a well-validated survey instrument, the standardized implementation and processing of the survey, and the addition of clinical data to the data obtained in the survey. As the CPQ-BC 2.0 focuses on experiences and events [1 ] during the patientʼs stay in the Breast Center, the results of the survey can be used to monitor whether patients also confirm that the Breast Center is adhering to the criteria for Breast Center accreditation. Although compliance with accreditation requirements is scrutinized during the accreditation audit, it is just as important to find out which services and information actually reach the patient, based on the patientʼs own perceptions. The data can also be used to show differences in real patient care based on patient characteristics and the characteristics of different Breast Centers and to highlight long-term trends in patient care. An attempt was made to explain the reasons for any deficits revealed by the survey by carrying out additional investigations, for example, by carrying out a survey among the senior management and staff of the Breast Centers (EBRU II study carried out by the IMVR, supported by the State Center for Health NRW [29 ], [30 ], [31 ]). Combining the perspective of patients with those of the management and staff working in the Breast Centers provides the opportunity to investigate correlations between the work situation and organization in the hospitals and the quality of care as perceived by the patients. It was found, for example, that the deployment of Breast Care nurses correlated with higher levels of patient satisfaction with nursing care [32 ]. Another finding was that patients treated in hospitals where physicians had very heavy workloads reported that they felt less supported by their physicians [31 ]. Such findings provide a further impetus for Breast Centers to continue improving their already high standards of care through organizational and staff development strategies.
In addition to the strengths of the survey there are also some limitations. Although the surveyed patients are highly representative with regard to sociodemographic characteristics and disease, it is not possible to exclude selection bias in favor of particularly satisfied patients. Because only breast cancer patients with a primary diagnosis of disease are included in the survey, the results do not allow any conclusions to be drawn about the quality of care offered to patients with recurrence. As the transfer of survey results by the IMVR ends with the findings being reported back to the Breast Centers and discussed at the benchmark workshop, it is not clear whether and how the results of the survey are used by individual surgical units and whether the findings contribute to improving patient care.
Conclusions
The use of regular surveys, the Cologne Patient Questionnaire for Breast Cancer 2.0 (CPQ-BC 2.0) and the benchmark-focused feedback aim to encourage organizational learning in Breast Centers. We hope that the findings of the survey and the workshops provide valuable information to the staff working in the Breast Centers and support the care they provide. The concept could also be used to encourage organizational learning in other healthcare organizations (e.g., in other cancer centers and rehabilitation clinics) and improve patient care there.
Acknowledgement
We would like to thank all of the Breast Centers and patients who participated in the survey. Our special thanks also go to the team of Markus Alich responsible for the survey in the IMVR.
